>>> kops: Building testing/kops 1.25.3-r0 (using abuild 3.10.0-r0) started Tue, 06 Dec 2022 14:22:26 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.25.0-r3) (2/3) Installing go (1.19.3-r0) (3/3) Installing .makedepends-kops (20221206.142228) Executing busybox-1.35.0-r29.trigger OK: 801 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 146 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 >>> kops: Fetching kops-1.25.3.tar.gz::https://github.com/kubernetes/kops/archive/v1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 6768k 0 6768k 0 0 5144k 0 --:--:-- 0:00:01 --:--:-- 5144k 100 12.4M 0 12.4M 0 0 5494k 0 --:--:-- 0:00:02 --:--:-- 5952k 100 16.8M 0 16.8M 0 0 5197k 0 --:--:-- 0:00:03 --:--:-- 5232k 100 21.9M 0 21.9M 0 0 5194k 0 --:--:-- 0:00:04 --:--:-- 5216k 100 25.5M 0 25.5M 0 0 4926k 0 --:--:-- 0:00:05 --:--:-- 4854k 100 30.2M 0 30.2M 0 0 4904k 0 --:--:-- 0:00:06 --:--:-- 4841k 100 31.1M 0 31.1M 0 0 4822k 0 --:--:-- 0:00:06 --:--:-- 4459k >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading k8s.io/klog/v2 v2.80.1 go: downloading google.golang.org/api v0.103.0 go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading k8s.io/api v0.25.4 go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/spf13/viper v1.14.0 go: downloading k8s.io/apimachinery v0.25.4 go: downloading go.uber.org/multierr v1.8.0 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/client-go v0.25.4 go: downloading k8s.io/klog v1.0.0 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading github.com/stretchr/testify v1.8.1 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading golang.org/x/crypto v0.3.0 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/kubectl v0.25.4 go: downloading golang.org/x/sync v0.1.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading k8s.io/component-base v0.25.4 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading golang.org/x/net v0.2.0 go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading golang.org/x/term v0.2.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/imdario/mergo v0.3.12 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading golang.org/x/sys v0.2.0 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading k8s.io/cloud-provider v0.25.4 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading google.golang.org/grpc v1.50.1 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading golang.org/x/text v0.4.0 go: downloading github.com/golang/protobuf v1.5.2 go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading github.com/json-iterator/go v1.1.12 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading go.opencensus.io v0.24.0 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I1206 14:24:39.078630 30929 addon.go:233] addon test wants to update all nodes I1206 14:24:39.080776 30929 addon.go:233] addon test wants to update worker nodes I1206 14:24:39.081124 30929 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I1206 14:24:39.081582 30929 addon.go:266] installing PKI for "test" I1206 14:24:40.663137 30929 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (2.52s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 2.658s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.073s ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN TestCreateClusterMinimal I1206 14:24:53.623682 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.623926 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.623970 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.624007 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.624048 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.624140 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.624158 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.624188 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.624215 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.624288 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.624330 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.683602 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.684308 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.684880 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.704194 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.704492 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.704564 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.704631 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.704697 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.704757 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.704808 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.704874 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.704922 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.705043 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.705107 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.747001 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.747206 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.747515 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.752510 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.752662 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.752698 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.752730 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.752762 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.752785 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.752802 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.752824 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.752842 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.752902 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.752926 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.786898 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.787048 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.787342 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.792341 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.792478 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.792511 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.792537 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.792568 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.792593 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.792615 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.792637 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.792654 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.792715 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.792740 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.838971 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.839141 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.839451 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.845313 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.845472 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.845509 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.845537 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.845566 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.845592 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.845611 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.845632 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.845649 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.845718 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.845743 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.895394 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.895633 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.896068 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.903351 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.903501 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.903537 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.903566 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.903597 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.903623 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.903637 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.903657 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.903673 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.903737 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.903762 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.936178 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.936332 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.936617 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.943836 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.944084 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.944157 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.944224 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.944294 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.944355 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.944403 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.944459 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.944510 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.944656 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.944726 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.978989 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:53.979277 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:53.979724 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:53.989704 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:53.989880 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:53.989913 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:53.989944 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:53.989976 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:53.990001 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:53.990027 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:53.990052 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:53.990069 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:53.990128 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:53.990154 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.021196 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.021288 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.021555 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.40s) === RUN TestCreateClusterHetzner I1206 14:24:54.027280 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.027487 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.027522 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.027551 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.027583 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.027634 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.027652 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.027673 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.027691 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.027748 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.027772 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1206 14:24:54.087633 35400 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1206 14:24:54.087697 35400 new_cluster.go:1279] Cloud Provider ID = hetzner I1206 14:24:54.094760 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.094967 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.095019 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.095049 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.095081 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.095117 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.095134 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.095155 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.095178 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.095235 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.095268 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.125571 35400 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.10s) === RUN TestCreateClusterOverride I1206 14:24:54.132037 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.132273 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.132316 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.132363 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.132405 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.132475 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.132503 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.132536 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.132561 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.132641 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.132681 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.228880 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.229347 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.229606 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.10s) === RUN TestCreateClusterComplex I1206 14:24:54.234932 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.235110 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.235141 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.235169 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.235200 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.235252 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.235273 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.235298 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.235315 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.235372 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.235397 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.318977 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.319145 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.319410 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.09s) === RUN TestCreateClusterHA I1206 14:24:54.326746 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.326924 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.326956 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.326985 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.327015 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.327065 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.327082 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.327104 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.327121 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.327177 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.327203 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.390701 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.391296 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.391322 35400 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1206 14:24:54.391337 35400 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1206 14:24:54.391819 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.405963 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.406197 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.406246 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.406292 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.406338 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.406384 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.406418 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.406455 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.406482 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.406566 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.406605 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.493929 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.494241 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.494257 35400 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1206 14:24:54.494267 35400 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1206 14:24:54.494560 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.18s) === RUN TestCreateClusterHAGCE I1206 14:24:54.506652 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.506841 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.506886 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.506925 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.506965 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.507037 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.507061 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.507092 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.507115 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.507193 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.507227 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.576990 35400 new_cluster.go:1279] Cloud Provider ID = gce I1206 14:24:54.577168 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.08s) === RUN TestCreateClusterGCE I1206 14:24:54.588851 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.589048 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.589093 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.589134 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.589178 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.589251 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.589276 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.589308 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.589333 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.589410 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.589442 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.618535 35400 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1206 14:24:54.618608 35400 new_cluster.go:1279] Cloud Provider ID = gce I1206 14:24:54.618680 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.04s) === RUN TestCreateClusterHASharedZone I1206 14:24:54.623934 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.624093 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.624129 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.624156 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.624185 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.624241 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.624259 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.624280 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.624297 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.624357 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.624379 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1206 14:24:54.663286 35400 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1206 14:24:54.663341 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.663540 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.663840 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.05s) === RUN TestCreateClusterHASharedZones I1206 14:24:54.670113 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.670263 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.670298 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.670328 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.670358 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.670409 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.670427 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.670449 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.670466 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.670524 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.670548 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W1206 14:24:54.707864 35400 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I1206 14:24:54.707924 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.708221 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.708235 35400 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1206 14:24:54.708584 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I1206 14:24:54.716705 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.716855 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.716890 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.716918 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.716949 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.716999 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.717014 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.717035 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.717051 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.717110 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.717133 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.756482 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.756742 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.756758 35400 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1206 14:24:54.757059 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.757089 35400 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I1206 14:24:54.763786 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.763952 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.763981 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.764009 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.764038 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.764086 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.764102 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.764123 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.764139 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.764193 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.764216 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.813962 35400 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I1206 14:24:54.814029 35400 new_cluster.go:1279] Cloud Provider ID = gce I1206 14:24:54.814107 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.06s) === RUN TestCreateClusterWithNGWSpecified I1206 14:24:54.820792 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.820947 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.820976 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.821004 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.821035 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.821086 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.821103 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.821126 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.821143 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.821198 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.821221 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.843396 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.843590 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.843606 35400 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1206 14:24:54.843879 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.843910 35400 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.03s) === RUN TestCreateClusterWithINGWSpecified I1206 14:24:54.850184 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.850373 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.850406 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.850432 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.850465 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.850516 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.850534 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.850559 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.850576 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.850632 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.850656 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.888398 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.888615 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:54.888631 35400 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1206 14:24:54.888975 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.889014 35400 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.05s) === RUN TestCreateClusterSharedVPC I1206 14:24:54.895263 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.895433 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.895474 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.895502 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.895532 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.895594 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.895612 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.895641 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.895657 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.895719 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.895743 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.967545 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:54.967707 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:54.967763 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:54.967803 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:54.967821 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:54.967867 35400 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I1206 14:24:54.968135 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.08s) === RUN TestCreateClusterSharedSubnets I1206 14:24:54.973328 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.973495 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.973533 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.973560 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.973589 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.973646 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.973663 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.973692 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.973707 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.973765 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:54.973788 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.039269 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.039318 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.039390 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.039525 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.039546 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.039570 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.039585 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.039872 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.07s) === RUN TestCreateClusterSharedSubnetsVpcLookup I1206 14:24:55.048004 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.048195 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.048237 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.048265 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.048326 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.048389 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.048407 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.048429 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.048446 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.048509 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.048540 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.090067 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.090149 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.090175 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.090251 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.090393 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.090419 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.090457 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.090475 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.090778 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.05s) === RUN TestCreateClusterPrivateSharedSubnets I1206 14:24:55.096087 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.096251 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.096291 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.096316 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.096348 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.096405 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.096422 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.096452 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.096467 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.096551 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.096577 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.131403 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.131446 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.131512 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.131530 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.131557 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.131738 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.131768 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.131789 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:24:55.131803 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I1206 14:24:55.132114 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.04s) === RUN TestCreateClusterIPv6 I1206 14:24:55.137648 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.137807 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.137843 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.137868 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.137896 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.137952 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.137970 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.138001 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.138018 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.138071 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.138095 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.176308 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.176484 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:55.176855 35400 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestCreateClusterDifferentAMIs I1206 14:24:55.184438 35400 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1206 14:24:55.184462 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.184619 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.184663 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.184700 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.184744 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.184804 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.184829 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.184863 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.184886 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.184973 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.185003 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.218831 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.218950 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:55.218971 35400 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I1206 14:24:55.219448 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.229007 35400 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.04s) === RUN TestCreateClusterKarpenter I1206 14:24:55.229143 35400 featureflag.go:160] FeatureFlag "Karpenter"=true I1206 14:24:55.229160 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.229345 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.229388 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.229433 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.229473 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.229542 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.229567 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.229596 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.229622 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.229702 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.229735 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I1206 14:24:55.329658 35400 new_cluster.go:1279] Cloud Provider ID = aws I1206 14:24:55.329902 35400 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I1206 14:24:55.329920 35400 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I1206 14:24:55.329932 35400 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I1206 14:24:55.330391 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.339661 35400 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.11s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I1206 14:24:55.340096 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.340277 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.340321 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.340399 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.340442 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.340482 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.340507 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.340542 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.340568 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.348748 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I1206 14:24:55.350461 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.350688 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.350729 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.350764 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.350809 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.350880 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.350907 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.350943 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.350969 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.368145 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:24:55.369680 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:55.369737 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1206 14:24:57.726493 35400 zones.go:93] ListHostedZonesPages { } I1206 14:24:59.716273 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1206 14:24:59.717206 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:24:59.721836 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:24:59.722263 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:24:59.723500 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:24:59.723537 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:24:59.723693 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:24:59.724489 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:24:59.727994 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:24:59.729012 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:59.729412 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:59.730553 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:59.732298 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:59.732456 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:24:59.732871 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:24:59.733112 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:24:59.770967 35400 target.go:112] Terraform output is in /tmp/test3936714652/out --- PASS: TestMinimal (4.42s) === RUN TestMinimal_v1_23 I1206 14:24:59.773463 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:59.773616 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:59.773653 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:59.773683 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:59.773716 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:59.773768 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:59.773786 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:59.773808 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:59.773827 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:59.788379 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1206 14:24:59.790260 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:24:59.790320 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.374764 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:00.495940 35400 executor.go:111] Tasks: 0 done / 92 total; 45 can run W1206 14:25:00.497138 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:00.497519 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:00.497759 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:00.497794 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:00.497727 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:00.497885 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:00.501999 35400 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1206 14:25:00.505902 35400 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1206 14:25:00.506325 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.506569 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.508025 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.508178 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.508252 35400 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1206 14:25:00.508655 35400 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1206 14:25:00.508836 35400 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1206 14:25:00.550425 35400 target.go:112] Terraform output is in /tmp/test3037400049/out --- PASS: TestMinimal_v1_23 (0.78s) === RUN TestMinimal_v1_24 I1206 14:25:00.552913 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:00.553061 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:00.553098 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:00.553130 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:00.553159 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:00.553213 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:00.553230 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:00.553252 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:00.553268 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:00.567738 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:00.569612 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:00.569669 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.022812 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:01.159914 35400 executor.go:111] Tasks: 0 done / 93 total; 46 can run W1206 14:25:01.160452 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:01.160570 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:01.160663 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:01.160617 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:01.160759 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:01.160980 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:01.163114 35400 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1206 14:25:01.167944 35400 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1206 14:25:01.168399 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.168732 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.169859 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.172331 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.172499 35400 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1206 14:25:01.172781 35400 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1206 14:25:01.172963 35400 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1206 14:25:01.228032 35400 target.go:112] Terraform output is in /tmp/test1128235347/out --- PASS: TestMinimal_v1_24 (0.68s) === RUN TestMinimal_v1_25 I1206 14:25:01.232561 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:01.232815 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:01.232879 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:01.232931 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:01.232987 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:01.233089 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:01.233120 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:01.233158 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:01.233190 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:01.258751 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:01.262297 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.262404 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.729129 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:01.865197 35400 executor.go:111] Tasks: 0 done / 93 total; 46 can run I1206 14:25:01.866520 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:01.866842 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:01.865644 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:01.866876 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:01.866986 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:01.868859 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:01.870416 35400 executor.go:111] Tasks: 46 done / 93 total; 19 can run I1206 14:25:01.875267 35400 executor.go:111] Tasks: 65 done / 93 total; 24 can run I1206 14:25:01.875600 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.875829 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.876853 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.878564 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.878692 35400 executor.go:111] Tasks: 89 done / 93 total; 2 can run I1206 14:25:01.879072 35400 executor.go:111] Tasks: 91 done / 93 total; 2 can run I1206 14:25:01.879344 35400 executor.go:111] Tasks: 93 done / 93 total; 0 can run I1206 14:25:01.931036 35400 target.go:112] Terraform output is in /tmp/test1295467086/out --- PASS: TestMinimal_v1_25 (0.70s) === RUN TestMinimal_v1_26 I1206 14:25:01.933670 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:01.933818 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:01.933863 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:01.933895 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:01.933930 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:01.933993 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:01.934010 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:01.934033 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:01.934051 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:01.948895 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:01.950939 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:01.951007 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:02.418645 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:02.579621 35400 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1206 14:25:02.580049 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:02.580073 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:02.580093 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:02.580194 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:02.580275 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:02.580439 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:02.582108 35400 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1206 14:25:02.585203 35400 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1206 14:25:02.585547 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:02.585753 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:02.586330 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:02.588411 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:02.588535 35400 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1206 14:25:02.588792 35400 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1206 14:25:02.588927 35400 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1206 14:25:02.636158 35400 target.go:112] Terraform output is in /tmp/test4056434100/out --- PASS: TestMinimal_v1_26 (0.70s) === RUN TestHetzner I1206 14:25:02.638739 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:03.208299 35400 executor.go:111] Tasks: 0 done / 50 total; 39 can run I1206 14:25:03.212810 35400 executor.go:111] Tasks: 39 done / 50 total; 5 can run I1206 14:25:03.213893 35400 executor.go:111] Tasks: 44 done / 50 total; 2 can run I1206 14:25:03.217729 35400 executor.go:111] Tasks: 46 done / 50 total; 4 can run I1206 14:25:03.219600 35400 executor.go:111] Tasks: 50 done / 50 total; 0 can run I1206 14:25:03.231536 35400 target.go:112] Terraform output is in /tmp/test1473900640/out --- PASS: TestHetzner (0.60s) === RUN TestNvidia I1206 14:25:03.236089 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:03.236367 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:03.236419 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:03.236469 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:03.236519 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:03.236604 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:03.236635 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:03.236675 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:03.236702 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:03.259977 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:03.262622 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.262697 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.306119 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:03.412927 35400 executor.go:111] Tasks: 0 done / 88 total; 44 can run I1206 14:25:03.413298 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:03.413460 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:03.413558 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:03.415213 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:03.413589 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:03.416052 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:03.419677 35400 executor.go:111] Tasks: 44 done / 88 total; 19 can run I1206 14:25:03.425155 35400 executor.go:111] Tasks: 63 done / 88 total; 21 can run I1206 14:25:03.425477 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.425756 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.427082 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.428038 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.428232 35400 executor.go:111] Tasks: 84 done / 88 total; 2 can run I1206 14:25:03.428693 35400 executor.go:111] Tasks: 86 done / 88 total; 2 can run I1206 14:25:03.428883 35400 executor.go:111] Tasks: 88 done / 88 total; 0 can run I1206 14:25:03.470529 35400 target.go:112] Terraform output is in /tmp/test2830918847/out --- PASS: TestNvidia (0.24s) === RUN TestMinimalGossip I1206 14:25:03.474575 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:03.474816 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:03.474867 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:03.474918 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:03.474988 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:03.475075 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:03.475106 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:03.475144 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:03.475172 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:03.504861 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.504998 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.605570 35400 executor.go:111] Tasks: 0 done / 86 total; 42 can run W1206 14:25:03.606096 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:03.606850 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:03.607624 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:03.611000 35400 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1206 14:25:03.615341 35400 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1206 14:25:03.615840 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.616104 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.616937 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.617782 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.617866 35400 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1206 14:25:03.618202 35400 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1206 14:25:03.618393 35400 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1206 14:25:03.651454 35400 target.go:112] Terraform output is in /tmp/test2935630322/out --- PASS: TestMinimalGossip (0.18s) === RUN TestMinimalGossipIRSA I1206 14:25:03.654173 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:03.654357 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:03.654394 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:03.654426 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:03.654461 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:03.654528 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:03.654563 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:03.654589 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:03.654608 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:03.675149 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.675409 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.842226 35400 executor.go:111] Tasks: 0 done / 89 total; 44 can run W1206 14:25:03.842595 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:03.842821 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:03.843164 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:03.846878 35400 executor.go:111] Tasks: 44 done / 89 total; 20 can run I1206 14:25:03.852145 35400 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1206 14:25:03.852401 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.852549 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.853069 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.853853 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:03.853943 35400 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1206 14:25:03.854532 35400 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1206 14:25:03.854815 35400 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1206 14:25:03.906676 35400 target.go:112] Terraform output is in /tmp/test3445961203/out --- PASS: TestMinimalGossipIRSA (0.26s) === RUN TestMinimalGCE I1206 14:25:03.909395 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:03.910464 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1206 14:25:04.352600 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:04.352714 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:04.352843 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1206 14:25:04.352888 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1206 14:25:04.352915 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:04.352947 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:04.353032 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:04.354788 35400 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1206 14:25:04.360540 35400 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1206 14:25:04.363825 35400 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1206 14:25:04.365406 35400 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1206 14:25:04.365806 35400 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1206 14:25:04.389171 35400 target.go:112] Terraform output is in /tmp/test1819279947/out --- PASS: TestMinimalGCE (0.48s) === RUN TestMinimalGCEPrivate I1206 14:25:04.392924 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:04.394833 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:04.772281 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:04.772377 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:04.772564 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W1206 14:25:04.772606 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W1206 14:25:04.772632 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1206 14:25:04.772667 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W1206 14:25:04.772740 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:04.774575 35400 executor.go:111] Tasks: 0 done / 66 total; 40 can run I1206 14:25:04.780914 35400 executor.go:111] Tasks: 40 done / 66 total; 18 can run I1206 14:25:04.784269 35400 executor.go:111] Tasks: 58 done / 66 total; 4 can run I1206 14:25:04.786497 35400 executor.go:111] Tasks: 62 done / 66 total; 3 can run I1206 14:25:04.787923 35400 executor.go:111] Tasks: 65 done / 66 total; 1 can run I1206 14:25:04.788075 35400 executor.go:111] Tasks: 66 done / 66 total; 0 can run I1206 14:25:04.817541 35400 target.go:112] Terraform output is in /tmp/test246077469/out --- PASS: TestMinimalGCEPrivate (0.43s) === RUN TestMinimalGCEInternalLoadBalancer I1206 14:25:04.821288 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:04.823305 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:04.969017 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:04.969066 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:04.969140 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W1206 14:25:04.969161 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W1206 14:25:04.969175 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1206 14:25:04.969195 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W1206 14:25:04.969235 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1206 14:25:04.969262 35400 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1206 14:25:04.970377 35400 executor.go:111] Tasks: 0 done / 68 total; 41 can run I1206 14:25:04.975859 35400 executor.go:111] Tasks: 41 done / 68 total; 17 can run W1206 14:25:04.980703 35400 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1206 14:25:04.980759 35400 executor.go:111] Tasks: 58 done / 68 total; 4 can run I1206 14:25:04.983619 35400 executor.go:111] Tasks: 62 done / 68 total; 3 can run I1206 14:25:04.984736 35400 executor.go:111] Tasks: 65 done / 68 total; 2 can run I1206 14:25:04.985015 35400 executor.go:111] Tasks: 67 done / 68 total; 1 can run I1206 14:25:04.985238 35400 executor.go:111] Tasks: 68 done / 68 total; 0 can run I1206 14:25:05.006365 35400 target.go:112] Terraform output is in /tmp/test2506509237/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.19s) === RUN TestMinimalGCELongClusterName I1206 14:25:05.008482 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:05.009550 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1206 14:25:05.208164 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:05.208332 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:05.208513 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1206 14:25:05.208555 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1206 14:25:05.208584 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1206 14:25:05.208630 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1206 14:25:05.208733 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:05.210531 35400 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1206 14:25:05.213154 35400 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1206 14:25:05.222407 35400 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1206 14:25:05.225389 35400 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1206 14:25:05.225591 35400 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1206 14:25:05.257375 35400 target.go:112] Terraform output is in /tmp/test3677803380/out --- PASS: TestMinimalGCELongClusterName (0.25s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I1206 14:25:05.261280 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:05.263274 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1206 14:25:05.493599 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:05.493693 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:05.493837 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1206 14:25:05.493905 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W1206 14:25:05.493934 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W1206 14:25:05.493960 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W1206 14:25:05.494035 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1206 14:25:05.494090 35400 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I1206 14:25:05.495978 35400 executor.go:111] Tasks: 0 done / 67 total; 42 can run I1206 14:25:05.498721 35400 executor.go:111] Tasks: 42 done / 67 total; 16 can run W1206 14:25:05.502745 35400 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I1206 14:25:05.502791 35400 executor.go:111] Tasks: 58 done / 67 total; 5 can run I1206 14:25:05.507264 35400 executor.go:111] Tasks: 63 done / 67 total; 2 can run I1206 14:25:05.507605 35400 executor.go:111] Tasks: 65 done / 67 total; 1 can run I1206 14:25:05.507933 35400 executor.go:111] Tasks: 66 done / 67 total; 1 can run I1206 14:25:05.509511 35400 executor.go:111] Tasks: 67 done / 67 total; 0 can run I1206 14:25:05.542574 35400 target.go:112] Terraform output is in /tmp/test2666873143/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.29s) === RUN TestHA I1206 14:25:05.548754 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:05.549019 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:05.549075 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:05.549177 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:05.549233 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:05.549275 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:05.549304 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:05.549339 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:05.549365 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:05.580905 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:05.584352 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.584447 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.584509 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.584568 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.622545 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:05.726037 35400 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1206 14:25:05.726434 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:05.726688 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:05.726754 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:05.726769 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:05.726926 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:05.727019 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:05.727059 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:05.727179 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:05.729267 35400 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1206 14:25:05.739066 35400 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1206 14:25:05.739579 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.739817 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.740036 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.740298 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.740691 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.742138 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.744406 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.744521 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:05.744654 35400 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1206 14:25:05.745085 35400 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1206 14:25:05.745380 35400 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1206 14:25:05.824559 35400 target.go:112] Terraform output is in /tmp/test138900137/out --- PASS: TestHA (0.28s) === RUN TestHighAvailabilityGCE I1206 14:25:05.829878 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:05.831959 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:06.025840 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:06.025917 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:06.025995 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1206 14:25:06.026035 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W1206 14:25:06.026053 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W1206 14:25:06.026069 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W1206 14:25:06.026112 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1206 14:25:06.026191 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups W1206 14:25:06.026236 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:06.027814 35400 executor.go:111] Tasks: 0 done / 83 total; 48 can run I1206 14:25:06.030028 35400 executor.go:111] Tasks: 48 done / 83 total; 20 can run I1206 14:25:06.035183 35400 executor.go:111] Tasks: 68 done / 83 total; 7 can run I1206 14:25:06.042885 35400 executor.go:111] Tasks: 75 done / 83 total; 5 can run I1206 14:25:06.044288 35400 executor.go:111] Tasks: 80 done / 83 total; 3 can run I1206 14:25:06.044538 35400 executor.go:111] Tasks: 83 done / 83 total; 0 can run I1206 14:25:06.087377 35400 target.go:112] Terraform output is in /tmp/test1882723214/out --- PASS: TestHighAvailabilityGCE (0.26s) === RUN TestComplex I1206 14:25:06.092065 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:06.092307 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:06.092362 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:06.092470 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:06.092522 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:06.092564 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:06.092596 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:06.092632 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:06.092720 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:06.126203 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:06.130936 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.131138 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.587557 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:06.788198 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1206 14:25:06.788538 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:06.788565 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:06.789108 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:06.789134 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:06.789217 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:06.791414 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:06.802036 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:06.802820 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.803090 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.804435 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.805940 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.806126 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:06.806648 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:06.806906 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1206 14:25:06.881987 35400 target.go:112] Terraform output is in /tmp/test1995608590/out I1206 14:25:06.886519 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:06.886781 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:06.886854 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:06.886932 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:06.887021 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:06.887088 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:06.887147 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:06.887216 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:06.887264 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:06.919641 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:06.923978 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.924092 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:06.971396 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:07.163228 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1206 14:25:07.164191 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:07.164337 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:07.164360 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:07.164420 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:07.165519 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:07.168883 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:07.174133 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:07.174513 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.174863 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.176055 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.178299 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.178444 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:07.178937 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:07.179215 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1206 14:25:07.255375 35400 target.go:112] Terraform output is in /tmp/test81326350/out --- PASS: TestComplex (1.17s) === RUN TestCompress I1206 14:25:07.260073 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:07.260339 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:07.260396 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:07.260446 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:07.260498 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:07.260586 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:07.260616 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:07.260651 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:07.260694 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:07.289382 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:07.293059 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.293180 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.331281 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:07.417600 35400 executor.go:111] Tasks: 0 done / 86 total; 42 can run I1206 14:25:07.418163 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:07.418225 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:07.418203 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:07.418376 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:07.418438 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:07.421006 35400 executor.go:111] Tasks: 42 done / 86 total; 19 can run I1206 14:25:07.425933 35400 executor.go:111] Tasks: 61 done / 86 total; 21 can run I1206 14:25:07.426218 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.426445 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.432431 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.434332 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.434535 35400 executor.go:111] Tasks: 82 done / 86 total; 2 can run I1206 14:25:07.434901 35400 executor.go:111] Tasks: 84 done / 86 total; 2 can run I1206 14:25:07.435071 35400 executor.go:111] Tasks: 86 done / 86 total; 0 can run I1206 14:25:07.488415 35400 target.go:112] Terraform output is in /tmp/test2212651271/out --- PASS: TestCompress (0.23s) === RUN TestExternalPolicies I1206 14:25:07.492416 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:07.492677 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:07.492737 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:07.492790 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:07.492842 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:07.492932 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:07.492962 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:07.492997 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:07.493026 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:07.521037 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:07.524438 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.524526 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.565818 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:07.667203 35400 executor.go:111] Tasks: 0 done / 100 total; 45 can run I1206 14:25:07.667439 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:07.667467 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:07.667556 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:07.667889 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:07.668163 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:07.668169 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:07.670807 35400 executor.go:111] Tasks: 45 done / 100 total; 20 can run I1206 14:25:07.676891 35400 executor.go:111] Tasks: 65 done / 100 total; 30 can run I1206 14:25:07.677292 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.677657 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.679854 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.681981 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.683424 35400 executor.go:111] Tasks: 95 done / 100 total; 3 can run I1206 14:25:07.683925 35400 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1206 14:25:07.684136 35400 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1206 14:25:07.749209 35400 target.go:112] Terraform output is in /tmp/test182269910/out --- PASS: TestExternalPolicies (0.26s) === RUN TestMinimalIPv6 I1206 14:25:07.753317 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:07.753567 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:07.753627 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:07.753680 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:07.753736 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:07.753827 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:07.753857 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:07.753895 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:07.753926 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:07.779584 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:07.783148 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.783246 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:07.851572 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:08.015471 35400 executor.go:111] Tasks: 0 done / 106 total; 45 can run W1206 14:25:08.015760 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:08.016094 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:08.016270 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:08.016380 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:08.016415 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:08.016508 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:08.018166 35400 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1206 14:25:08.024649 35400 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1206 14:25:08.025010 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.025191 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.025786 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.026917 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.026996 35400 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1206 14:25:08.027538 35400 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1206 14:25:08.027834 35400 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1206 14:25:08.087085 35400 target.go:112] Terraform output is in /tmp/test3480678855/out --- PASS: TestMinimalIPv6 (0.34s) === RUN TestMinimalIPv6Private I1206 14:25:08.091650 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:08.091975 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:08.092046 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:08.092111 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:08.092173 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:08.092268 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:08.092300 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:08.092342 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:08.092373 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:08.121546 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1206 14:25:08.125184 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.125281 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.165203 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:08.347636 35400 executor.go:111] Tasks: 0 done / 124 total; 46 can run I1206 14:25:08.348127 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:08.348162 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:08.348307 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:08.348258 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:08.348496 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:08.350686 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:08.355014 35400 executor.go:111] Tasks: 46 done / 124 total; 23 can run I1206 14:25:08.361600 35400 executor.go:111] Tasks: 69 done / 124 total; 36 can run I1206 14:25:08.362266 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.362405 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.363507 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.365268 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.365409 35400 executor.go:111] Tasks: 105 done / 124 total; 11 can run I1206 14:25:08.366119 35400 executor.go:111] Tasks: 116 done / 124 total; 8 can run I1206 14:25:08.366500 35400 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1206 14:25:08.443260 35400 target.go:112] Terraform output is in /tmp/test150636234/out --- PASS: TestMinimalIPv6Private (0.36s) === RUN TestMinimalIPv6Calico I1206 14:25:08.447751 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:08.448013 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:08.448081 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:08.448201 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:08.448261 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:08.448355 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:08.448387 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:08.448429 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:08.448461 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:08.474926 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:08.478373 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.478474 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:08.523838 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:08.991290 35400 executor.go:111] Tasks: 0 done / 109 total; 47 can run W1206 14:25:08.991845 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:08.992258 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:08.992285 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:08.992377 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:08.992304 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:08.992539 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:08.994762 35400 executor.go:111] Tasks: 47 done / 109 total; 21 can run I1206 14:25:09.000199 35400 executor.go:111] Tasks: 68 done / 109 total; 31 can run I1206 14:25:09.001562 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.001921 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.003804 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.007023 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.007180 35400 executor.go:111] Tasks: 99 done / 109 total; 5 can run I1206 14:25:09.007678 35400 executor.go:111] Tasks: 104 done / 109 total; 5 can run I1206 14:25:09.008031 35400 executor.go:111] Tasks: 109 done / 109 total; 0 can run I1206 14:25:09.072924 35400 target.go:112] Terraform output is in /tmp/test1912570322/out --- PASS: TestMinimalIPv6Calico (0.63s) === RUN TestMinimalIPv6Cilium I1206 14:25:09.078366 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:09.078611 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:09.078663 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:09.078713 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:09.078763 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:09.078853 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:09.078882 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:09.078918 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:09.078947 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:09.103993 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:09.107846 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.107944 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.147213 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:09.362882 35400 executor.go:111] Tasks: 0 done / 107 total; 46 can run W1206 14:25:09.364702 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:09.365253 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:09.366663 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:09.367706 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:09.367811 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:09.367876 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:09.369594 35400 executor.go:111] Tasks: 46 done / 107 total; 21 can run I1206 14:25:09.374308 35400 executor.go:111] Tasks: 67 done / 107 total; 30 can run I1206 14:25:09.374685 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.374881 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.376316 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.380031 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.380362 35400 executor.go:111] Tasks: 97 done / 107 total; 5 can run I1206 14:25:09.382226 35400 executor.go:111] Tasks: 102 done / 107 total; 5 can run I1206 14:25:09.382816 35400 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1206 14:25:09.446990 35400 target.go:112] Terraform output is in /tmp/test1467563454/out --- PASS: TestMinimalIPv6Cilium (0.37s) === RUN TestMinimalWarmPool I1206 14:25:09.451495 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:09.451762 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:09.451828 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:09.451881 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:09.451938 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:09.452037 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:09.452072 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:09.452113 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:09.452144 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:09.477259 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:09.480904 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.481003 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.520312 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:09.729589 35400 executor.go:111] Tasks: 0 done / 89 total; 45 can run I1206 14:25:09.730579 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:09.731479 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:09.731651 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:09.732994 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:09.733367 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:09.733543 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:09.737450 35400 executor.go:111] Tasks: 45 done / 89 total; 19 can run I1206 14:25:09.742899 35400 executor.go:111] Tasks: 64 done / 89 total; 21 can run I1206 14:25:09.743477 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.743681 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.744904 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.746091 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.746213 35400 executor.go:111] Tasks: 85 done / 89 total; 2 can run I1206 14:25:09.746571 35400 executor.go:111] Tasks: 87 done / 89 total; 2 can run I1206 14:25:09.746765 35400 executor.go:111] Tasks: 89 done / 89 total; 0 can run I1206 14:25:09.801487 35400 target.go:112] Terraform output is in /tmp/test1644095552/out --- PASS: TestMinimalWarmPool (0.35s) === RUN TestMinimalEtcd I1206 14:25:09.806099 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:09.806356 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:09.806421 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:09.806475 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:09.806533 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:09.806635 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:09.806664 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:09.806701 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:09.806735 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:09.833187 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:09.836476 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.836578 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.875566 35400 zones.go:93] ListHostedZonesPages { } W1206 14:25:09.970631 35400 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W1206 14:25:09.970695 35400 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W1206 14:25:09.972504 35400 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I1206 14:25:09.985467 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1206 14:25:09.985881 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:09.986104 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:09.986211 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:09.986233 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:09.986317 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:09.986320 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:09.993007 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:09.999100 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:09.999510 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:09.999767 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.000895 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.002664 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.002797 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:10.003235 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:10.003431 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:10.056169 35400 target.go:112] Terraform output is in /tmp/test2906984947/out --- PASS: TestMinimalEtcd (0.25s) === RUN TestMinimalGp3 I1206 14:25:10.060254 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:10.060508 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:10.060567 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:10.060620 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:10.060675 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:10.060775 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:10.060805 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:10.060842 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:10.060874 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:10.086135 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:10.089355 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.089450 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.128457 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:10.223916 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1206 14:25:10.224416 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:10.224570 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:10.224746 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:10.224737 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:10.224769 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:10.224828 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:10.226418 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:10.233462 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:10.233700 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.233869 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.234988 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.235204 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.235283 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:10.235560 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:10.235705 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:10.288683 35400 target.go:112] Terraform output is in /tmp/test3839161128/out --- PASS: TestMinimalGp3 (0.23s) === RUN TestMinimalLongClusterName I1206 14:25:10.292896 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:10.293175 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:10.293234 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:10.293357 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:10.293439 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:10.293489 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:10.293536 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:10.293594 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:10.293632 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:10.317346 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:10.320165 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.320251 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1206 14:25:10.358469 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:10.462355 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1206 14:25:10.463156 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:10.463376 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:10.463436 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:10.463418 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:10.463574 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:10.463579 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:10.465785 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:10.470951 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:10.471164 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.471727 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.472237 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.474157 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.474289 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:10.474631 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:10.474840 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:10.532214 35400 target.go:112] Terraform output is in /tmp/test1327192579/out --- PASS: TestMinimalLongClusterName (0.24s) === RUN TestExistingSG I1206 14:25:10.536345 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:10.536598 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:10.536661 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:10.536714 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:10.536771 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:10.536866 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:10.536896 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:10.536934 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:10.536965 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:10.574235 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:10.578131 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.578235 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.578319 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.578386 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.616807 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:10.734745 35400 executor.go:111] Tasks: 0 done / 146 total; 53 can run I1206 14:25:10.735543 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:10.735643 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:10.735784 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:10.735888 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:10.735923 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:10.736021 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:10.736050 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:10.736137 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:10.737555 35400 executor.go:111] Tasks: 53 done / 146 total; 26 can run I1206 14:25:10.747242 35400 executor.go:111] Tasks: 79 done / 146 total; 58 can run I1206 14:25:10.747756 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.748155 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.748372 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.748675 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.749288 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.750224 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.750731 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.751018 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.751145 35400 executor.go:111] Tasks: 137 done / 146 total; 5 can run I1206 14:25:10.751740 35400 executor.go:111] Tasks: 142 done / 146 total; 4 can run I1206 14:25:10.751996 35400 executor.go:111] Tasks: 146 done / 146 total; 0 can run I1206 14:25:10.850119 35400 target.go:112] Terraform output is in /tmp/test345362846/out --- PASS: TestExistingSG (0.32s) === RUN TestBastionAdditionalUserData I1206 14:25:10.855341 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:10.855606 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:10.855673 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:10.855787 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:10.855844 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:10.855893 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:10.855922 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:10.855966 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:10.855997 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:10.885359 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:10.888550 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.888637 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.888697 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:10.926431 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:11.034343 35400 executor.go:111] Tasks: 0 done / 122 total; 48 can run W1206 14:25:11.034980 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.034966 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:11.035086 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:11.035099 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.035351 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:11.035377 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:11.037818 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:11.042902 35400 executor.go:111] Tasks: 48 done / 122 total; 29 can run I1206 14:25:11.048427 35400 executor.go:111] Tasks: 77 done / 122 total; 36 can run I1206 14:25:11.048902 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.049370 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.049645 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.050414 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.050724 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.052206 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.052343 35400 executor.go:111] Tasks: 113 done / 122 total; 5 can run I1206 14:25:11.053017 35400 executor.go:111] Tasks: 118 done / 122 total; 4 can run I1206 14:25:11.053337 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1206 14:25:11.130464 35400 target.go:112] Terraform output is in /tmp/test1441710654/out --- PASS: TestBastionAdditionalUserData (0.28s) === RUN TestPrivateWeave I1206 14:25:11.135000 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:11.135247 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:11.135305 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:11.135357 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:11.135411 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:11.135504 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:11.135534 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:11.135568 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:11.135597 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:11.163165 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:11.166115 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.166200 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.166257 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.205022 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:11.337773 35400 executor.go:111] Tasks: 0 done / 120 total; 47 can run I1206 14:25:11.338052 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:11.338106 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.338094 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:11.338199 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:11.338584 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:11.338737 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.338880 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:11.341467 35400 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1206 14:25:11.346697 35400 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1206 14:25:11.346932 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.347116 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.347229 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.347473 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.348441 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.349747 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.349870 35400 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1206 14:25:11.351360 35400 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1206 14:25:11.351954 35400 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1206 14:25:11.431909 35400 target.go:112] Terraform output is in /tmp/test2513229526/out --- PASS: TestPrivateWeave (0.30s) === RUN TestPrivateFlannel I1206 14:25:11.436347 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:11.436583 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:11.436640 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:11.436743 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:11.436792 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:11.436834 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:11.436862 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:11.436898 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:11.436928 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:11.464637 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:11.468084 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.468180 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.468243 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.507538 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:11.710672 35400 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1206 14:25:11.711212 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.711367 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:11.711452 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.711410 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:11.711510 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:11.711693 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:11.711715 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:11.713594 35400 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1206 14:25:11.719087 35400 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1206 14:25:11.719424 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.719604 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.719760 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.719973 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.721011 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.722611 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.722744 35400 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1206 14:25:11.724291 35400 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1206 14:25:11.724805 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:11.803300 35400 target.go:112] Terraform output is in /tmp/test105864463/out --- PASS: TestPrivateFlannel (0.37s) === RUN TestPrivateCalico I1206 14:25:11.808242 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:11.808500 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:11.808554 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:11.808652 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:11.808705 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:11.808745 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:11.808775 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:11.808817 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:11.808847 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:11.837677 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:11.841133 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.841221 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.841280 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:11.881359 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:12.341093 35400 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1206 14:25:12.341451 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:12.341503 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:12.341623 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:12.341874 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:12.342074 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:12.342136 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:12.342168 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:12.344503 35400 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1206 14:25:12.349901 35400 executor.go:111] Tasks: 78 done / 127 total; 40 can run I1206 14:25:12.350152 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.350617 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.350880 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.351862 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.353082 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.356840 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.356973 35400 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1206 14:25:12.357454 35400 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1206 14:25:12.357718 35400 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1206 14:25:12.436079 35400 target.go:112] Terraform output is in /tmp/test2858563167/out --- PASS: TestPrivateCalico (0.63s) === RUN TestPrivateCilium I1206 14:25:12.441713 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:12.441976 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:12.442039 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:12.442096 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:12.442152 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:12.442254 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:12.442284 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:12.442323 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:12.442355 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:12.470819 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:12.474099 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.474185 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.474240 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.513487 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:12.669060 35400 executor.go:111] Tasks: 0 done / 120 total; 47 can run W1206 14:25:12.669391 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:12.669824 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:12.669867 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:12.669961 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:12.670124 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:12.670195 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:12.670290 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:12.672845 35400 executor.go:111] Tasks: 47 done / 120 total; 28 can run I1206 14:25:12.678239 35400 executor.go:111] Tasks: 75 done / 120 total; 36 can run I1206 14:25:12.678623 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.680303 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.680469 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.680710 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.682454 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.685679 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.685823 35400 executor.go:111] Tasks: 111 done / 120 total; 5 can run I1206 14:25:12.686290 35400 executor.go:111] Tasks: 116 done / 120 total; 4 can run I1206 14:25:12.686644 35400 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1206 14:25:12.761487 35400 target.go:112] Terraform output is in /tmp/test1557306905/out --- PASS: TestPrivateCilium (0.32s) === RUN TestPrivateCilium2 I1206 14:25:12.766006 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:12.766259 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:12.766317 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:12.766419 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:12.766470 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:12.766518 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:12.766547 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:12.766583 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:12.766609 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:12.793478 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:12.799299 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.799403 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.799469 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:12.838711 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:13.497889 35400 executor.go:111] Tasks: 0 done / 124 total; 51 can run W1206 14:25:13.498462 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:13.498552 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:13.498550 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:13.498820 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:13.498889 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:13.498864 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:13.499004 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:13.505419 35400 executor.go:111] Tasks: 51 done / 124 total; 28 can run I1206 14:25:13.511078 35400 executor.go:111] Tasks: 79 done / 124 total; 36 can run I1206 14:25:13.511542 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.512005 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.512231 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.512490 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.513241 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.514187 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.514316 35400 executor.go:111] Tasks: 115 done / 124 total; 5 can run I1206 14:25:13.514758 35400 executor.go:111] Tasks: 120 done / 124 total; 4 can run I1206 14:25:13.515182 35400 executor.go:111] Tasks: 124 done / 124 total; 0 can run I1206 14:25:13.593716 35400 target.go:112] Terraform output is in /tmp/test328378677/out --- PASS: TestPrivateCilium2 (0.83s) === RUN TestPrivateCiliumAdvanced I1206 14:25:13.601091 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:13.601340 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:13.601400 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:13.601452 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:13.601508 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:13.601626 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:13.601661 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:13.601700 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:13.601732 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:13.634534 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:13.637860 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.637948 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.638003 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.677860 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:13.831523 35400 executor.go:111] Tasks: 0 done / 126 total; 53 can run W1206 14:25:13.832087 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:13.832283 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:13.832367 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:13.832553 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:13.832565 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:13.832601 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:13.832717 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:13.835062 35400 executor.go:111] Tasks: 53 done / 126 total; 28 can run I1206 14:25:13.839903 35400 executor.go:111] Tasks: 81 done / 126 total; 36 can run I1206 14:25:13.840482 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.840631 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.840843 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.841011 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.841574 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.843059 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.843194 35400 executor.go:111] Tasks: 117 done / 126 total; 5 can run I1206 14:25:13.843616 35400 executor.go:111] Tasks: 122 done / 126 total; 4 can run I1206 14:25:13.843965 35400 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1206 14:25:13.927072 35400 target.go:112] Terraform output is in /tmp/test863691256/out --- PASS: TestPrivateCiliumAdvanced (0.33s) === RUN TestPrivateCanal I1206 14:25:13.932043 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:13.932289 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:13.932349 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:13.932456 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:13.932511 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:13.932561 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:13.932591 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:13.932629 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:13.932659 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:13.960599 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:13.964130 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.964228 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:13.964294 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.002790 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:14.473117 35400 executor.go:111] Tasks: 0 done / 123 total; 50 can run W1206 14:25:14.473603 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:14.473732 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:14.473767 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:14.473827 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:14.474017 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:14.474041 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:14.474111 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:14.476608 35400 executor.go:111] Tasks: 50 done / 123 total; 28 can run I1206 14:25:14.485320 35400 executor.go:111] Tasks: 78 done / 123 total; 36 can run I1206 14:25:14.485606 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.485780 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.486064 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.486391 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.487400 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.488638 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:14.488777 35400 executor.go:111] Tasks: 114 done / 123 total; 5 can run I1206 14:25:14.489212 35400 executor.go:111] Tasks: 119 done / 123 total; 4 can run I1206 14:25:14.489550 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:14.565302 35400 target.go:112] Terraform output is in /tmp/test2151981458/out --- PASS: TestPrivateCanal (0.64s) === RUN TestPrivateKopeio I1206 14:25:14.570806 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:14.571065 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:14.571128 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:14.571185 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:14.571240 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:14.571343 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:14.571374 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:14.571413 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:14.571445 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:14.602547 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:14.607389 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.607501 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.607589 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.646064 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:14.766822 35400 executor.go:111] Tasks: 0 done / 126 total; 47 can run W1206 14:25:14.767294 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:14.767358 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:14.767542 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:14.767635 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:14.767978 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:14.768021 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:14.768159 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:14.770518 35400 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1206 14:25:14.776221 35400 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1206 14:25:14.776884 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.777064 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.777348 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.777520 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.778146 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.779212 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.779336 35400 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1206 14:25:14.779802 35400 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1206 14:25:14.780075 35400 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1206 14:25:14.860751 35400 target.go:112] Terraform output is in /tmp/test3117010272/out --- PASS: TestPrivateKopeio (0.29s) === RUN TestUnmanaged I1206 14:25:14.865250 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:14.865546 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:14.865605 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:14.865660 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:14.865715 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:14.865808 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:14.865842 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:14.865880 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:14.865909 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:14.895457 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:14.898682 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.898764 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.898818 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:14.936813 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:15.044734 35400 executor.go:111] Tasks: 0 done / 108 total; 45 can run W1206 14:25:15.045408 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.045554 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:15.045613 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:15.045694 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.045620 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:15.045758 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:15.045780 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:15.049720 35400 executor.go:111] Tasks: 45 done / 108 total; 25 can run I1206 14:25:15.058035 35400 executor.go:111] Tasks: 70 done / 108 total; 31 can run I1206 14:25:15.058469 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.058640 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.058713 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.058811 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.059953 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.060998 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.061128 35400 executor.go:111] Tasks: 101 done / 108 total; 4 can run I1206 14:25:15.061530 35400 executor.go:111] Tasks: 105 done / 108 total; 3 can run I1206 14:25:15.061869 35400 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1206 14:25:15.135791 35400 target.go:112] Terraform output is in /tmp/test2164016447/out --- PASS: TestUnmanaged (0.27s) === RUN TestPrivateSharedSubnet I1206 14:25:15.139984 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:15.140239 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:15.140302 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:15.140353 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:15.140409 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:15.140511 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:15.140553 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:15.140593 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:15.140625 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:15.169110 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:15.172897 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.172990 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.173047 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.211352 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:15.319950 35400 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1206 14:25:15.320342 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:15.320410 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.320374 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:15.320470 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:15.320644 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:15.320903 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.320886 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:15.322484 35400 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1206 14:25:15.323005 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:15.327408 35400 executor.go:111] Tasks: 69 done / 107 total; 31 can run I1206 14:25:15.327624 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.328930 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.329668 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.330337 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.332793 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.334295 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.334449 35400 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1206 14:25:15.334842 35400 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1206 14:25:15.335133 35400 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1206 14:25:15.401228 35400 target.go:112] Terraform output is in /tmp/test479088308/out --- PASS: TestPrivateSharedSubnet (0.27s) === RUN TestPrivateSharedIP I1206 14:25:15.405637 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:15.405875 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:15.405927 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:15.405979 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:15.406031 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:15.406117 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:15.406147 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:15.406186 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:15.406214 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:15.434558 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:15.437900 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.437994 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.438055 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.476865 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:15.584274 35400 executor.go:111] Tasks: 0 done / 116 total; 45 can run W1206 14:25:15.585001 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.585101 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:15.585149 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.585141 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:15.585275 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:15.585328 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:15.585490 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.590977 35400 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1206 14:25:15.591688 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:15.596872 35400 executor.go:111] Tasks: 71 done / 116 total; 36 can run I1206 14:25:15.597245 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.597620 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.597829 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.597907 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.598675 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.600403 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:15.600531 35400 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1206 14:25:15.600944 35400 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1206 14:25:15.601254 35400 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1206 14:25:15.676910 35400 target.go:112] Terraform output is in /tmp/test1216570672/out --- PASS: TestPrivateSharedIP (0.28s) === RUN TestPrivateDns1 I1206 14:25:15.681188 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:15.681444 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:15.681511 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:15.681564 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:15.681639 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:15.681748 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:15.681779 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:15.681817 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:15.681850 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:15.712621 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.712706 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.712767 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.865977 35400 executor.go:111] Tasks: 0 done / 120 total; 46 can run W1206 14:25:15.866502 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.866560 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:15.866963 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:15.866863 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:15.874220 35400 executor.go:111] Tasks: 46 done / 120 total; 24 can run I1206 14:25:15.874864 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I1206 14:25:15.874925 35400 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I1206 14:25:15.875073 35400 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I1206 14:25:15.875095 35400 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I1206 14:25:15.875183 35400 executor.go:111] Tasks: 70 done / 120 total; 35 can run I1206 14:25:15.875595 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.875874 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.878726 35400 executor.go:111] Tasks: 105 done / 120 total; 5 can run W1206 14:25:15.878811 35400 api.go:53] PageSize not implemented W1206 14:25:15.878870 35400 api.go:53] PageSize not implemented I1206 14:25:15.884071 35400 executor.go:111] Tasks: 110 done / 120 total; 6 can run I1206 14:25:15.884364 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.884563 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.885516 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.886995 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:15.887124 35400 executor.go:111] Tasks: 116 done / 120 total; 2 can run I1206 14:25:15.887611 35400 executor.go:111] Tasks: 118 done / 120 total; 2 can run I1206 14:25:15.887878 35400 executor.go:111] Tasks: 120 done / 120 total; 0 can run I1206 14:25:15.976075 35400 target.go:112] Terraform output is in /tmp/test258836208/out --- PASS: TestPrivateDns1 (0.30s) === RUN TestPrivateDns2 I1206 14:25:15.980418 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:15.980676 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:15.980742 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:15.980797 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:15.980853 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:15.980953 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:15.980983 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:15.981021 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:15.981053 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:16.013128 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.013233 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.013297 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.147639 35400 executor.go:111] Tasks: 0 done / 116 total; 44 can run W1206 14:25:16.148172 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:16.148595 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:16.148766 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:16.148743 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:16.151853 35400 executor.go:111] Tasks: 44 done / 116 total; 22 can run I1206 14:25:16.152434 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I1206 14:25:16.152428 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:16.152591 35400 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I1206 14:25:16.152916 35400 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I1206 14:25:16.153027 35400 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I1206 14:25:16.153256 35400 executor.go:111] Tasks: 66 done / 116 total; 35 can run I1206 14:25:16.155924 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.156961 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.159608 35400 executor.go:111] Tasks: 101 done / 116 total; 5 can run W1206 14:25:16.159788 35400 api.go:53] PageSize not implemented W1206 14:25:16.159900 35400 api.go:53] PageSize not implemented I1206 14:25:16.165077 35400 executor.go:111] Tasks: 106 done / 116 total; 6 can run I1206 14:25:16.165375 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.165568 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.166448 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.168211 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.168389 35400 executor.go:111] Tasks: 112 done / 116 total; 2 can run I1206 14:25:16.168773 35400 executor.go:111] Tasks: 114 done / 116 total; 2 can run I1206 14:25:16.168965 35400 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1206 14:25:16.243349 35400 target.go:112] Terraform output is in /tmp/test2821233046/out --- PASS: TestPrivateDns2 (0.27s) === RUN TestDiscoveryFeatureGate I1206 14:25:16.247927 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:16.248174 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:16.248232 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:16.248375 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:16.248430 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:16.248472 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:16.248501 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:16.248537 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:16.248569 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:16.279087 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:16.282826 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.282933 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.321562 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:16.539604 35400 executor.go:111] Tasks: 0 done / 99 total; 51 can run I1206 14:25:16.540150 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:16.540197 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:16.540188 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:16.540328 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:16.540277 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:16.540487 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:16.547588 35400 executor.go:111] Tasks: 51 done / 99 total; 23 can run I1206 14:25:16.553784 35400 executor.go:111] Tasks: 74 done / 99 total; 21 can run I1206 14:25:16.554276 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.554496 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.555726 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.556951 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.557069 35400 executor.go:111] Tasks: 95 done / 99 total; 2 can run I1206 14:25:16.557384 35400 executor.go:111] Tasks: 97 done / 99 total; 2 can run I1206 14:25:16.557595 35400 executor.go:111] Tasks: 99 done / 99 total; 0 can run I1206 14:25:16.615950 35400 target.go:112] Terraform output is in /tmp/test2448122972/out --- PASS: TestDiscoveryFeatureGate (0.37s) === RUN TestVFSServiceAccountIssuerDiscovery I1206 14:25:16.621380 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:16.621656 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:16.621732 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:16.621804 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:16.621865 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:16.621963 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:16.621994 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:16.622032 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:16.622064 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:16.650731 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:16.653834 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.653933 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.692333 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:16.801963 35400 executor.go:111] Tasks: 0 done / 90 total; 45 can run W1206 14:25:16.802231 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:16.802330 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:16.802766 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:16.802801 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:16.802889 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:16.803188 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:16.805709 35400 executor.go:111] Tasks: 45 done / 90 total; 20 can run I1206 14:25:16.810885 35400 executor.go:111] Tasks: 65 done / 90 total; 21 can run I1206 14:25:16.811294 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.811519 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.812647 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.813591 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.813714 35400 executor.go:111] Tasks: 86 done / 90 total; 2 can run I1206 14:25:16.814025 35400 executor.go:111] Tasks: 88 done / 90 total; 2 can run I1206 14:25:16.814221 35400 executor.go:111] Tasks: 90 done / 90 total; 0 can run I1206 14:25:16.868773 35400 target.go:112] Terraform output is in /tmp/test3340039047/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.25s) === RUN TestAWSLBController I1206 14:25:16.873142 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:16.873386 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:16.873442 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:16.873492 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:16.873540 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:16.873626 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:16.873657 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:16.873694 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:16.873722 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:16.902373 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:16.905971 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.906076 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:16.944710 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:17.551138 35400 executor.go:111] Tasks: 0 done / 96 total; 49 can run W1206 14:25:17.552173 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:17.552156 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:17.552321 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:17.554154 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:17.554193 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:17.554346 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:17.560306 35400 executor.go:111] Tasks: 49 done / 96 total; 22 can run I1206 14:25:17.565760 35400 executor.go:111] Tasks: 71 done / 96 total; 21 can run I1206 14:25:17.565976 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.566318 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.567550 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.569118 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.569244 35400 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1206 14:25:17.569596 35400 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1206 14:25:17.569962 35400 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1206 14:25:17.629629 35400 target.go:112] Terraform output is in /tmp/test43723571/out --- PASS: TestAWSLBController (0.77s) === RUN TestManyAddons I1206 14:25:17.640851 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:17.641120 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:17.641189 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:17.641246 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:17.641307 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:17.641412 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:17.641444 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:17.641485 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:17.641519 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:17.667928 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:17.672281 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.672386 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:17.713118 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:18.511149 35400 executor.go:111] Tasks: 0 done / 95 total; 50 can run I1206 14:25:18.511779 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:18.511971 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:18.512063 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:18.512099 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:18.512235 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:18.512205 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:18.517916 35400 executor.go:111] Tasks: 50 done / 95 total; 19 can run I1206 14:25:18.523605 35400 executor.go:111] Tasks: 69 done / 95 total; 22 can run I1206 14:25:18.523840 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.524122 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.524733 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.526781 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.527022 35400 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1206 14:25:18.527615 35400 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1206 14:25:18.527827 35400 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1206 14:25:18.580415 35400 target.go:112] Terraform output is in /tmp/test755276433/out --- PASS: TestManyAddons (0.95s) === RUN TestManyAddonsCCMIRSA I1206 14:25:18.592814 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:18.593087 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:18.593142 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:18.593253 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:18.593308 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:18.593349 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:18.593377 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:18.593412 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:18.593440 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:18.621501 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:18.625323 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.625426 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:18.663932 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:19.531414 35400 executor.go:111] Tasks: 0 done / 111 total; 59 can run W1206 14:25:19.532346 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:19.532393 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:19.532616 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:19.532678 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:19.532714 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:19.532811 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:19.538821 35400 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1206 14:25:19.544156 35400 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1206 14:25:19.544572 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.544797 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.546827 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.547845 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.547978 35400 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1206 14:25:19.548329 35400 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1206 14:25:19.548621 35400 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1206 14:25:19.620321 35400 target.go:112] Terraform output is in /tmp/test1731595781/out --- PASS: TestManyAddonsCCMIRSA (1.04s) === RUN TestManyAddonsCCMIRSA23 I1206 14:25:19.629555 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:19.629813 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:19.629880 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:19.629994 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:19.630051 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:19.630097 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:19.630128 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:19.630166 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:19.630203 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:19.657358 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1206 14:25:19.660568 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.660651 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:19.700783 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:20.568537 35400 executor.go:111] Tasks: 0 done / 112 total; 60 can run W1206 14:25:20.569086 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:20.569219 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:20.569576 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:20.569757 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:20.569721 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:20.570330 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:20.576192 35400 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1206 14:25:20.581831 35400 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1206 14:25:20.582381 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.583444 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.586649 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.588946 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.589083 35400 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1206 14:25:20.589522 35400 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1206 14:25:20.589925 35400 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1206 14:25:20.659869 35400 target.go:112] Terraform output is in /tmp/test708910381/out --- PASS: TestManyAddonsCCMIRSA23 (1.04s) === RUN TestManyAddonsCCMIRSA24 I1206 14:25:20.669431 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:20.669684 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:20.669749 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:20.669858 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:20.669912 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:20.669961 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:20.669991 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:20.670029 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:20.670062 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:20.697906 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:20.701799 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.702278 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:20.741302 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:21.590649 35400 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1206 14:25:21.591392 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:21.591515 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:21.591580 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:21.591573 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:21.591750 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:21.591886 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:21.600270 35400 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1206 14:25:21.606000 35400 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1206 14:25:21.606523 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.606834 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.607760 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.609502 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.609633 35400 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1206 14:25:21.609953 35400 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1206 14:25:21.610221 35400 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1206 14:25:21.680767 35400 target.go:112] Terraform output is in /tmp/test1400439513/out --- PASS: TestManyAddonsCCMIRSA24 (1.02s) === RUN TestManyAddonsCCMIRSA25 I1206 14:25:21.690872 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:21.691123 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:21.691182 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:21.691290 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:21.691344 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:21.691387 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:21.691417 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:21.691457 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:21.691487 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:21.718251 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:21.721538 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.721659 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:21.762721 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:22.625101 35400 executor.go:111] Tasks: 0 done / 112 total; 60 can run I1206 14:25:22.625778 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:22.625824 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:22.625942 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:22.626359 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:22.627362 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:22.627924 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:22.635300 35400 executor.go:111] Tasks: 60 done / 112 total; 26 can run I1206 14:25:22.640726 35400 executor.go:111] Tasks: 86 done / 112 total; 22 can run I1206 14:25:22.641002 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.641264 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.642231 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.646123 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.648479 35400 executor.go:111] Tasks: 108 done / 112 total; 2 can run I1206 14:25:22.649004 35400 executor.go:111] Tasks: 110 done / 112 total; 2 can run I1206 14:25:22.649311 35400 executor.go:111] Tasks: 112 done / 112 total; 0 can run I1206 14:25:22.716162 35400 target.go:112] Terraform output is in /tmp/test2682460985/out --- PASS: TestManyAddonsCCMIRSA25 (1.04s) === RUN TestManyAddonsCCMIRSA26 I1206 14:25:22.726020 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:22.726283 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:22.726352 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:22.726407 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:22.726463 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:22.726565 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:22.726596 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:22.726638 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:22.726670 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:22.754106 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:22.757681 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.757775 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:22.799632 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:23.699308 35400 executor.go:111] Tasks: 0 done / 111 total; 59 can run I1206 14:25:23.699710 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:23.699773 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:23.699741 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:23.699986 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:23.700109 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:23.703005 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:23.708770 35400 executor.go:111] Tasks: 59 done / 111 total; 26 can run I1206 14:25:23.715230 35400 executor.go:111] Tasks: 85 done / 111 total; 22 can run I1206 14:25:23.715822 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.716939 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.719528 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.722739 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.722901 35400 executor.go:111] Tasks: 107 done / 111 total; 2 can run I1206 14:25:23.723228 35400 executor.go:111] Tasks: 109 done / 111 total; 2 can run I1206 14:25:23.723578 35400 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1206 14:25:23.790598 35400 target.go:112] Terraform output is in /tmp/test4198107301/out --- PASS: TestManyAddonsCCMIRSA26 (1.07s) === RUN TestCCM I1206 14:25:23.799907 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:23.800169 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:23.800234 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:23.800287 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:23.800343 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:23.800460 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:23.800491 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:23.800531 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:23.800562 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:23.827396 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:23.830954 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.831049 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:23.869529 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:24.659061 35400 executor.go:111] Tasks: 0 done / 96 total; 51 can run I1206 14:25:24.659613 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:24.659778 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:24.659824 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:24.659893 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:24.659827 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:24.659964 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:24.669047 35400 executor.go:111] Tasks: 51 done / 96 total; 19 can run I1206 14:25:24.675254 35400 executor.go:111] Tasks: 70 done / 96 total; 22 can run I1206 14:25:24.675637 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.675928 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.676901 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.681598 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.681765 35400 executor.go:111] Tasks: 92 done / 96 total; 2 can run I1206 14:25:24.682069 35400 executor.go:111] Tasks: 94 done / 96 total; 2 can run I1206 14:25:24.682255 35400 executor.go:111] Tasks: 96 done / 96 total; 0 can run I1206 14:25:24.735205 35400 target.go:112] Terraform output is in /tmp/test626947558/out --- PASS: TestCCM (0.95s) === RUN TestExternalDNS I1206 14:25:24.747402 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:24.747661 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:24.747722 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:24.747777 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:24.747830 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:24.747917 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:24.747949 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:24.747987 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:24.748016 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:24.772428 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:24.775454 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.775547 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.814673 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:24.920566 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1206 14:25:24.920828 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:24.920956 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:24.920995 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:24.921138 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:24.922026 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:24.921991 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:24.923619 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:24.928855 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:24.929309 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.929479 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.930387 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.931836 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:24.931961 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:24.932298 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:24.932507 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:24.985638 35400 target.go:112] Terraform output is in /tmp/test365296647/out --- PASS: TestExternalDNS (0.24s) === RUN TestExternalDNSIRSA I1206 14:25:24.989580 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:24.989827 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:24.989884 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:24.989936 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:24.989989 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:24.990079 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:24.990132 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:24.990167 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:24.990196 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:25.015112 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:25.018202 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.018325 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.057494 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:25.179330 35400 executor.go:111] Tasks: 0 done / 92 total; 46 can run W1206 14:25:25.181043 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:25.182768 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:25.183826 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:25.184014 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:25.184168 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:25.185595 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:25.187319 35400 executor.go:111] Tasks: 46 done / 92 total; 21 can run I1206 14:25:25.193277 35400 executor.go:111] Tasks: 67 done / 92 total; 21 can run I1206 14:25:25.193633 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.193787 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.195199 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.196205 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.196324 35400 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1206 14:25:25.196671 35400 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1206 14:25:25.196865 35400 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1206 14:25:25.253750 35400 target.go:112] Terraform output is in /tmp/test1784495325/out --- PASS: TestExternalDNSIRSA (0.27s) === RUN TestKarpenter I1206 14:25:25.258450 35400 featureflag.go:160] FeatureFlag "Karpenter"=true I1206 14:25:25.258508 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:25.258765 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:25.258823 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:25.258873 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:25.258927 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:25.259022 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:25.259052 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:25.259091 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:25.259122 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:25.289847 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:25.293590 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.293679 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.293742 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.337553 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:25.526332 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.526438 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.526521 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.630451 35400 executor.go:111] Tasks: 0 done / 108 total; 53 can run I1206 14:25:25.631004 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:25.631153 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:25.631288 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:25.631402 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:25.631331 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:25.631471 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:25.633209 35400 executor.go:111] Tasks: 53 done / 108 total; 26 can run I1206 14:25:25.638785 35400 executor.go:111] Tasks: 79 done / 108 total; 25 can run I1206 14:25:25.639193 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.639475 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.639750 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.640057 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.640630 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.641072 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.641330 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.642014 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.642162 35400 executor.go:111] Tasks: 104 done / 108 total; 2 can run I1206 14:25:25.642496 35400 executor.go:111] Tasks: 106 done / 108 total; 2 can run I1206 14:25:25.642667 35400 executor.go:111] Tasks: 108 done / 108 total; 0 can run I1206 14:25:25.716098 35400 target.go:112] Terraform output is in /tmp/test641608186/out I1206 14:25:25.722565 35400 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.46s) === RUN TestSharedSubnet I1206 14:25:25.722712 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:25.722966 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:25.723032 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:25.723089 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:25.723144 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:25.723307 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:25.723345 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:25.723385 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:25.723417 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:25.749146 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:25.752482 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.752584 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.791415 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:25.887484 35400 executor.go:111] Tasks: 0 done / 80 total; 42 can run W1206 14:25:25.887784 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:25.887908 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:25.888117 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:25.888178 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:25.888201 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:25.888284 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:25.890956 35400 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1206 14:25:25.891646 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:25.897239 35400 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1206 14:25:25.897677 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.897805 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.898858 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.900043 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.900165 35400 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1206 14:25:25.900507 35400 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1206 14:25:25.900719 35400 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1206 14:25:25.951740 35400 target.go:112] Terraform output is in /tmp/test2773343117/out --- PASS: TestSharedSubnet (0.23s) === RUN TestSharedVPC I1206 14:25:25.955728 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:25.955984 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:25.956042 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:25.956092 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:25.956145 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:25.956234 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:25.956265 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:25.956301 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:25.956331 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:25.980415 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:25.983483 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:25.983581 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.022338 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:26.130092 35400 executor.go:111] Tasks: 0 done / 84 total; 42 can run W1206 14:25:26.130498 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.130532 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:26.130558 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:26.130656 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:26.130788 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.131086 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:26.133421 35400 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1206 14:25:26.133642 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:26.139065 35400 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1206 14:25:26.139265 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.139608 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.140492 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.143984 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.144508 35400 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1206 14:25:26.145258 35400 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1206 14:25:26.145804 35400 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1206 14:25:26.196465 35400 target.go:112] Terraform output is in /tmp/test2310239003/out --- PASS: TestSharedVPC (0.24s) === RUN TestExistingIAM I1206 14:25:26.200425 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:26.200678 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:26.200740 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:26.200847 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:26.200906 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:26.200948 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:26.200977 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:26.201013 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:26.201041 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:26.233194 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:26.239832 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.239942 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.240009 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.240074 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.279520 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:26.397732 35400 executor.go:111] Tasks: 0 done / 101 total; 51 can run W1206 14:25:26.398005 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.398289 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:26.398321 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W1206 14:25:26.398406 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.398414 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:26.398439 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:26.398686 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.398794 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:26.400465 35400 executor.go:111] Tasks: 51 done / 101 total; 15 can run I1206 14:25:26.405841 35400 executor.go:111] Tasks: 66 done / 101 total; 27 can run I1206 14:25:26.406223 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.406364 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.406513 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.407313 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.409103 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.412965 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.413076 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.413385 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:26.413572 35400 executor.go:111] Tasks: 93 done / 101 total; 4 can run I1206 14:25:26.413937 35400 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1206 14:25:26.414227 35400 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1206 14:25:26.488899 35400 target.go:112] Terraform output is in /tmp/test2789903182/out --- PASS: TestExistingIAM (0.30s) === RUN TestPhaseNetwork I1206 14:25:26.496349 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:26.496655 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:26.496723 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:26.496788 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:26.496844 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:26.496893 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:26.496923 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:26.496963 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:26.496993 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:26.526678 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:26.530094 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.530202 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.530268 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.569281 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:26.676763 35400 executor.go:111] Tasks: 0 done / 119 total; 46 can run I1206 14:25:26.677126 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:26.677229 35400 executor.go:111] Tasks: 46 done / 119 total; 28 can run I1206 14:25:26.678843 35400 executor.go:111] Tasks: 74 done / 119 total; 36 can run I1206 14:25:26.679479 35400 executor.go:111] Tasks: 110 done / 119 total; 5 can run I1206 14:25:26.680094 35400 executor.go:111] Tasks: 115 done / 119 total; 4 can run I1206 14:25:26.680534 35400 executor.go:111] Tasks: 119 done / 119 total; 0 can run I1206 14:25:26.691287 35400 target.go:112] Terraform output is in /tmp/test4064592759/out --- PASS: TestPhaseNetwork (0.20s) === RUN TestExternalLoadBalancer I1206 14:25:26.692104 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:26.692350 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:26.692408 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:26.692456 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:26.692513 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:26.692604 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:26.692634 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:26.692671 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:26.692704 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:26.719571 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:26.722856 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.722962 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.764204 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:26.875659 35400 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1206 14:25:26.876145 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:26.876264 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:26.876331 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:26.876416 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:26.876439 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:26.876542 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:26.878793 35400 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1206 14:25:26.888484 35400 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1206 14:25:26.888942 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.889198 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.890289 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.891427 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.891638 35400 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1206 14:25:26.892205 35400 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1206 14:25:26.892412 35400 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1206 14:25:26.947933 35400 target.go:112] Terraform output is in /tmp/test1928386862/out --- PASS: TestExternalLoadBalancer (0.26s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I1206 14:25:26.952362 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:26.952612 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:26.952664 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:26.952716 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:26.952766 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:26.952858 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:26.952889 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:26.952925 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:26.952953 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:26.985666 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:26.988990 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989073 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989126 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989181 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989229 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989274 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:26.989320 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.033647 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:27.155061 35400 executor.go:111] Tasks: 0 done / 111 total; 53 can run W1206 14:25:27.155344 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.155545 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:27.155857 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:27.155902 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:27.157039 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:27.155798 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:27.159979 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:27.159877 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.163027 35400 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1206 14:25:27.171008 35400 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1206 14:25:27.171283 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.171919 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.172180 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.172463 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.175160 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.176143 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.176529 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.177603 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.177765 35400 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1206 14:25:27.178858 35400 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1206 14:25:27.179183 35400 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1206 14:25:27.258955 35400 target.go:112] Terraform output is in /tmp/test22773282/out --- PASS: TestMixedInstancesASG (0.31s) === RUN TestMixedInstancesSpotASG I1206 14:25:27.264885 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:27.265178 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:27.265252 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:27.265310 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:27.265367 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:27.265463 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:27.265492 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:27.265530 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:27.265565 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:27.304452 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:27.308106 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308214 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308300 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308369 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308423 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308473 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.308567 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.348224 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:27.465550 35400 executor.go:111] Tasks: 0 done / 111 total; 53 can run I1206 14:25:27.466303 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" W1206 14:25:27.466344 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.466346 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:27.466450 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:27.466531 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:27.466681 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.466751 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:27.466846 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.469956 35400 executor.go:111] Tasks: 53 done / 111 total; 23 can run I1206 14:25:27.478933 35400 executor.go:111] Tasks: 76 done / 111 total; 27 can run I1206 14:25:27.479291 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.479530 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.479679 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.480014 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.480704 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.481770 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.482177 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.482326 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:27.482450 35400 executor.go:111] Tasks: 103 done / 111 total; 4 can run I1206 14:25:27.482975 35400 executor.go:111] Tasks: 107 done / 111 total; 4 can run I1206 14:25:27.483303 35400 executor.go:111] Tasks: 111 done / 111 total; 0 can run I1206 14:25:27.567925 35400 target.go:112] Terraform output is in /tmp/test854067675/out --- PASS: TestMixedInstancesSpotASG (0.31s) === RUN TestAdditionalObjects I1206 14:25:27.573847 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:27.574101 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:27.574156 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:27.574209 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:27.574261 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:27.574371 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:27.574404 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:27.574442 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:27.574472 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:27.605351 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I1206 14:25:27.608814 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.608921 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.648964 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:27.809614 35400 executor.go:111] Tasks: 0 done / 92 total; 45 can run I1206 14:25:27.809857 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:27.809879 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:27.809959 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:27.810193 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.811405 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:27.812068 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:27.816993 35400 executor.go:111] Tasks: 45 done / 92 total; 19 can run I1206 14:25:27.822936 35400 executor.go:111] Tasks: 64 done / 92 total; 24 can run I1206 14:25:27.823213 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.823524 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.824650 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.826341 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I1206 14:25:27.826473 35400 executor.go:111] Tasks: 88 done / 92 total; 2 can run I1206 14:25:27.826956 35400 executor.go:111] Tasks: 90 done / 92 total; 2 can run I1206 14:25:27.827217 35400 executor.go:111] Tasks: 92 done / 92 total; 0 can run I1206 14:25:27.886668 35400 target.go:112] Terraform output is in /tmp/test1877824281/out --- PASS: TestAdditionalObjects (0.32s) === RUN TestContainerd I1206 14:25:27.890985 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:27.891247 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:27.891312 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:27.891370 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:27.891428 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:27.891533 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:27.891566 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:27.891627 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:27.891666 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:27.915872 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:27.919022 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:27.919121 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:27.959167 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:28.038455 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1206 14:25:28.038947 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:28.039191 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.039247 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:28.039289 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:28.039442 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:28.039260 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.041985 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:28.047626 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:28.047948 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.048203 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.049146 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.050663 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.050870 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:28.051242 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:28.051417 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:28.102961 35400 target.go:112] Terraform output is in /tmp/test2211578274/out --- PASS: TestContainerd (0.22s) === RUN TestContainerdCustom I1206 14:25:28.107127 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:28.107383 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:28.107436 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:28.107488 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:28.107542 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:28.107629 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:28.107664 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:28.107700 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:28.107730 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:28.133351 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:28.136739 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.136834 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.176212 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:28.283038 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run W1206 14:25:28.283598 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.283699 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:28.283730 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:28.283816 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:28.284103 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.284078 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:28.286491 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:28.292015 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:28.292431 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.292632 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.294564 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.299537 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.299680 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:28.300009 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:28.300197 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:28.349677 35400 target.go:112] Terraform output is in /tmp/test1387644374/out --- PASS: TestContainerdCustom (0.25s) === RUN TestDockerCustom I1206 14:25:28.353687 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:28.353940 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:28.353998 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:28.354050 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:28.354103 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:28.354195 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:28.354225 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:28.354260 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:28.354291 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:28.382827 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:28.386310 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.386411 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.424731 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:28.526869 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1206 14:25:28.527369 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:28.527413 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:28.527648 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W1206 14:25:28.527731 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.527644 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:28.527560 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:28.533211 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:28.538176 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:28.538428 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.538739 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.540128 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.541050 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:28.541173 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:28.541486 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:28.541705 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:28.594192 35400 target.go:112] Terraform output is in /tmp/test1656310363/out --- PASS: TestDockerCustom (0.24s) === RUN TestAPIServerNodes I1206 14:25:28.598610 35400 featureflag.go:160] FeatureFlag "APIServerNodes"=true I1206 14:25:28.598642 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:28.599639 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:28.599742 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:28.599826 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:28.599914 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:28.600037 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:28.600129 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:28.600219 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:28.600267 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:28.629042 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I1206 14:25:28.632318 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:28.632413 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:28.632474 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.096212 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:29.261696 35400 executor.go:111] Tasks: 0 done / 100 total; 47 can run W1206 14:25:29.262162 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:29.262287 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:29.262326 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:29.265057 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:29.262683 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:29.265449 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:29.265590 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:29.270181 35400 executor.go:111] Tasks: 47 done / 100 total; 24 can run I1206 14:25:29.275720 35400 executor.go:111] Tasks: 71 done / 100 total; 23 can run I1206 14:25:29.276189 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.276326 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.276577 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.277588 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.277861 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.278998 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.279125 35400 executor.go:111] Tasks: 94 done / 100 total; 3 can run I1206 14:25:29.279509 35400 executor.go:111] Tasks: 97 done / 100 total; 3 can run I1206 14:25:29.279742 35400 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1206 14:25:29.346038 35400 target.go:112] Terraform output is in /tmp/test3761892471/out I1206 14:25:29.350742 35400 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (0.75s) === RUN TestNTHQueueProcessor I1206 14:25:29.350865 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:29.351115 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:29.351169 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:29.351217 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:29.351281 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:29.351371 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:29.351401 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:29.351438 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:29.351468 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:29.375963 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1206 14:25:29.379103 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.379197 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:29.856421 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:29.988722 35400 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1206 14:25:29.989429 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W1206 14:25:29.989647 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:29.989471 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:29.990122 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:29.990160 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:29.990277 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:29.991879 35400 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1206 14:25:30.001407 35400 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1206 14:25:30.001945 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.002223 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.003168 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.004969 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.005061 35400 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1206 14:25:30.005418 35400 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1206 14:25:30.005619 35400 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1206 14:25:30.062253 35400 target.go:112] Terraform output is in /tmp/test2738792427/out --- PASS: TestNTHQueueProcessor (0.72s) === RUN TestCustomIRSA I1206 14:25:30.067103 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:30.067352 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:30.067404 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:30.067453 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:30.067502 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:30.067590 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:30.067620 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:30.067655 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:30.067680 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:30.095918 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1206 14:25:30.099377 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.099462 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.140568 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:30.681800 35400 executor.go:111] Tasks: 0 done / 100 total; 50 can run W1206 14:25:30.682464 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:30.682621 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:30.682737 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:30.682804 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:30.682781 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:30.682907 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:30.688633 35400 executor.go:111] Tasks: 50 done / 100 total; 25 can run I1206 14:25:30.697023 35400 executor.go:111] Tasks: 75 done / 100 total; 21 can run I1206 14:25:30.697242 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.697621 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.698346 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.699839 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.699960 35400 executor.go:111] Tasks: 96 done / 100 total; 2 can run I1206 14:25:30.700258 35400 executor.go:111] Tasks: 98 done / 100 total; 2 can run I1206 14:25:30.700509 35400 executor.go:111] Tasks: 100 done / 100 total; 0 can run I1206 14:25:30.761338 35400 target.go:112] Terraform output is in /tmp/test4266796812/out --- PASS: TestCustomIRSA (0.70s) === RUN TestClusterNameDigit I1206 14:25:30.768657 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:30.768919 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:30.768983 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:30.769094 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:30.769151 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:30.769198 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:30.769226 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:30.769264 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:30.769297 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:30.796450 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1206 14:25:30.800202 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.800300 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.839386 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:30.948478 35400 executor.go:111] Tasks: 0 done / 95 total; 47 can run W1206 14:25:30.948797 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target W1206 14:25:30.949274 35400 warmpool.go:124] ASG warm pool is not supported by the terraform target I1206 14:25:30.949581 35400 dnszone.go:242] Check for existing route53 zone to re-use with name "" I1206 14:25:30.949648 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:30.949628 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:30.949758 35400 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I1206 14:25:30.951237 35400 executor.go:111] Tasks: 47 done / 95 total; 23 can run I1206 14:25:30.959066 35400 executor.go:111] Tasks: 70 done / 95 total; 21 can run I1206 14:25:30.960322 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.960707 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.961628 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.963067 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:30.963188 35400 executor.go:111] Tasks: 91 done / 95 total; 2 can run I1206 14:25:30.963493 35400 executor.go:111] Tasks: 93 done / 95 total; 2 can run I1206 14:25:30.963687 35400 executor.go:111] Tasks: 95 done / 95 total; 0 can run I1206 14:25:31.022062 35400 target.go:112] Terraform output is in /tmp/test2491794131/out --- PASS: TestClusterNameDigit (0.26s) === RUN TestLifecycleMinimalAWS I1206 14:25:31.026581 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:31.026830 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:31.026886 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:31.026940 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:31.026994 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:31.027037 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:31.027066 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:31.027101 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:31.027127 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:31.027195 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I1206 14:25:31.051091 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:31.054038 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.054162 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1206 14:25:31.093039 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:31.198918 35400 iamrole.go:89] ListRoles: { } I1206 14:25:31.200633 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1206 14:25:31.201152 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.201405 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:31.201366 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I1206 14:25:31.201504 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:31.201510 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1206 14:25:31.201532 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.201705 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.201793 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I1206 14:25:31.201821 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:31.201901 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I1206 14:25:31.201964 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.202218 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:31.202395 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:31.202469 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:31.202529 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:31.202557 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:31.202601 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.202718 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" }] }] } I1206 14:25:31.202881 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:31.203008 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:31.203056 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.203100 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.203212 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:31.204283 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:31.204517 35400 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1206 14:25:31.204457 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1206 14:25:31.204707 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I1206 14:25:31.204741 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1206 14:25:31.204839 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:31.204892 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1206 14:25:31.204965 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1206 14:25:31.204947 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.205005 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1206 14:25:31.205111 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:31.205157 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] } I1206 14:25:31.205245 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I1206 14:25:31.205231 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I1206 14:25:31.205352 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:31.205409 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:31.205465 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:31.205517 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.205641 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1206 14:25:31.205786 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.206011 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1206 14:25:31.206288 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.206494 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.206580 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:31.206624 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:31.206670 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:31.206724 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }], VpcId: "vpc-1" } I1206 14:25:31.206818 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:31.206869 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:31.206919 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:31.207318 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1206 14:25:31.210247 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:31.210513 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.210647 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.210829 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:31.210970 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.211180 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.211329 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I1206 14:25:31.211557 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:31.211719 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.211887 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.212094 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:31.212183 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.212308 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.212495 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.212620 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I1206 14:25:31.212798 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:31.212867 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:31.212937 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1206 14:25:31.213121 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.213292 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.213479 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.213653 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.213829 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I1206 14:25:31.214005 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:31.214177 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I1206 14:25:31.214346 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.215450 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:31.215778 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:31.215930 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:31.215991 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:31.216046 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:31.216133 35400 dns.go:238] Pre-creating DNS records I1206 14:25:31.216180 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:31.221571 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:31.226849 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.227009 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I1206 14:25:31.266174 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:31.368973 35400 iamrole.go:89] ListRoles: { } I1206 14:25:31.370394 35400 executor.go:111] Tasks: 0 done / 87 total; 43 can run I1206 14:25:31.370832 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.373388 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:31.373425 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:31.373506 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.373840 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.374090 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.374384 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:31.374496 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:31.374514 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:31.374775 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.377424 35400 executor.go:111] Tasks: 43 done / 87 total; 19 can run I1206 14:25:31.377598 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I1206 14:25:31.377710 35400 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I1206 14:25:31.377683 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.377950 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.378108 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.378286 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:31.378344 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.378450 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I1206 14:25:31.378635 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:31.378699 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I1206 14:25:31.378861 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:31.378911 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:31.384519 35400 executor.go:111] Tasks: 62 done / 87 total; 21 can run I1206 14:25:31.384732 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.384911 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.385316 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.385461 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.385629 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.385872 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.386109 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:31.388883 35400 executor.go:111] Tasks: 83 done / 87 total; 2 can run I1206 14:25:31.389119 35400 executor.go:111] Tasks: 85 done / 87 total; 2 can run I1206 14:25:31.389334 35400 executor.go:111] Tasks: 87 done / 87 total; 0 can run I1206 14:25:31.392364 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:31.392429 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:31.392470 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:31.392506 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:31.392539 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:31.392578 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:31.392613 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:31.392647 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:31.392683 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:31.392720 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:31.392756 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:31.392799 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:31.392839 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:31.392873 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:31.392905 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:31.392941 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:31.392974 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:31.393010 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:31.393046 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:31.393084 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:31.393120 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:31.393152 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:31.393185 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:31.393222 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:31.393259 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:31.393294 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:31.394611 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:31.394642 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:31.394686 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.394737 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1206 14:25:31.394793 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.394849 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.394898 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.394972 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.395023 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.395067 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I1206 14:25:31.395114 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.395164 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I1206 14:25:31.395207 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W1206 14:25:31.395253 35400 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W1206 14:25:31.395288 35400 api.go:53] PageSize not implemented I1206 14:25:31.395311 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:31.395344 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:31.395363 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:31.395394 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:31.395412 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:31.395467 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:31.395496 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:31.395544 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:31.395584 35400 iamrole.go:89] ListRoles: { } I1206 14:25:31.395637 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:31.395684 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:31.395720 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:31.396234 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal.example.com ok I1206 14:25:31.396294 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I1206 14:25:31.396352 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:31.396420 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I1206 14:25:31.396465 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1206 14:25:31.396516 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I1206 14:25:31.396455 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:31.396574 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:31.396451 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.minimal.example.com. ok I1206 14:25:31.396684 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1206 14:25:31.396913 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:31.396940 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:31.396982 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:31.397027 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I1206 14:25:31.397102 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:31.397153 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1206 14:25:31.397346 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:31.397360 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:31.397424 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I1206 14:25:31.397486 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:31.397490 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I1206 14:25:31.397592 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I1206 14:25:31.397611 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I1206 14:25:31.397655 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I1206 14:25:31.397700 35400 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I1206 14:25:31.397715 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I1206 14:25:31.397743 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I1206 14:25:31.397817 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:31.397909 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:31.397987 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:31.414054 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.39s) === RUN TestLifecycleMinimalOpenstack I1206 14:25:31.414404 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I1206 14:25:31.425250 35400 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:31.648218 35400 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1206 14:25:31.649639 35400 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1206 14:25:31.669847 35400 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1206 14:25:31.679580 35400 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1206 14:25:31.718979 35400 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1206 14:25:31.723266 35400 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1206 14:25:31.726563 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:31.726600 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:31.726851 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1206 14:25:31.731813 35400 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:31.927643 35400 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1206 14:25:31.927698 35400 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1206 14:25:31.950340 35400 executor.go:111] Tasks: 0 done / 96 total; 42 can run I1206 14:25:31.954643 35400 executor.go:111] Tasks: 42 done / 96 total; 45 can run I1206 14:25:31.966658 35400 executor.go:111] Tasks: 87 done / 96 total; 5 can run I1206 14:25:31.969296 35400 executor.go:111] Tasks: 92 done / 96 total; 4 can run W1206 14:25:31.972606 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:31.972941 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:31.973347 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1206 14:25:31.974959 35400 executor.go:111] Tasks: 96 done / 96 total; 0 can run Network:566a714a-5506-4948-8a6e-21d434f3cfac ok Port:81cd2a99-f0f3-4e9e-bb17-064ab2e76505 ok Instance:c178181b-535a-4a05-a559-a72658e0574c ok Port:212afc04-e946-4d64-a0ae-0e6edf2e08be ok Instance:1b018fcb-d324-49fa-8c62-8483d9a5ea33 ok Port:430b73f6-e883-487c-ad0a-a608b1025b4f ok SecurityGroup:67216970-fa68-4a74-bf37-d91da999ae8e ok ServerGroup:d5e5befe-eaf9-4e5a-a24d-59226ee33f36 ok ServerGroup:fa5bcd5e-c83b-4522-909d-d1bec9127091 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:eee85248-9d40-40f0-8266-3efdbd271a67 ok Subnet:46415af1-e398-4c91-a1b8-29ba5eb98098 ok SecurityGroup:e5691e5c-8c4a-4078-85a3-16dfe2fa123e ok SecurityGroup:78ba64f4-112e-4439-bc4b-5c6eed591bda ok --- PASS: TestLifecycleMinimalOpenstack (1.67s) === RUN TestLifecycleMinimalGCE I1206 14:25:33.083666 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I1206 14:25:33.085564 35400 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1206 14:25:33.320538 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:33.320632 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:33.320747 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1206 14:25:33.320784 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1206 14:25:33.320809 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:33.320842 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:33.320919 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:33.322740 35400 executor.go:111] Tasks: 0 done / 64 total; 41 can run W1206 14:25:33.325974 35400 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I1206 14:25:33.326030 35400 executor.go:111] Tasks: 40 done / 64 total; 18 can run I1206 14:25:33.331275 35400 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1206 14:25:33.334002 35400 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1206 14:25:33.334256 35400 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1206 14:25:33.334359 35400 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W1206 14:25:33.560748 35400 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W1206 14:25:33.560844 35400 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W1206 14:25:33.560942 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W1206 14:25:33.560979 35400 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W1206 14:25:33.561005 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:33.561040 35400 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W1206 14:25:33.561117 35400 autoscalinggroup.go:122] enabling storage-rw for etcd backups I1206 14:25:33.562920 35400 executor.go:111] Tasks: 0 done / 64 total; 41 can run I1206 14:25:33.566137 35400 executor.go:111] Tasks: 41 done / 64 total; 17 can run I1206 14:25:33.571510 35400 executor.go:111] Tasks: 58 done / 64 total; 4 can run I1206 14:25:33.574171 35400 executor.go:111] Tasks: 62 done / 64 total; 2 can run I1206 14:25:33.574346 35400 executor.go:111] Tasks: 64 done / 64 total; 0 can run I1206 14:25:33.575983 35400 gce.go:98] Scanning zones: [us-test1-a] I1206 14:25:33.576346 35400 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I1206 14:25:33.576371 35400 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:node-to-master-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1670336733 ok InstanceTemplate:nodes-minimal-gce-example-com-1670336733 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.49s) === RUN TestLifecycleFloatingIPOpenstack I1206 14:25:33.577862 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I1206 14:25:33.587670 35400 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:33.811513 35400 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1206 14:25:33.811556 35400 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1206 14:25:33.825642 35400 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1206 14:25:33.836882 35400 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1206 14:25:33.903427 35400 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1206 14:25:33.906743 35400 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1206 14:25:33.909345 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:33.909568 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:33.909689 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1206 14:25:33.914596 35400 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W1206 14:25:34.074914 35400 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W1206 14:25:34.074959 35400 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I1206 14:25:34.089910 35400 executor.go:111] Tasks: 0 done / 98 total; 43 can run I1206 14:25:34.093389 35400 executor.go:111] Tasks: 43 done / 98 total; 45 can run I1206 14:25:34.105562 35400 executor.go:111] Tasks: 88 done / 98 total; 6 can run I1206 14:25:34.107821 35400 executor.go:111] Tasks: 94 done / 98 total; 4 can run W1206 14:25:34.111559 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:34.111776 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W1206 14:25:34.111931 35400 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I1206 14:25:34.113782 35400 executor.go:111] Tasks: 98 done / 98 total; 0 can run Instance:81deedf4-5213-488e-94a3-e15405e844ec ok Instance:e6c0fa40-0c64-4b90-8fca-7c517aab8ef2 ok Network:11fbcd4c-255b-43f2-801f-50d1660f28c7 ok Port:33687fc7-4e3e-4658-a39c-d4e543cc05f7 ok Port:83d7da92-f5f1-403f-a291-5c5e20b655c6 ok Port:0eee9edb-0fa8-4ddd-8656-47223309deb7 ok ServerGroup:fad9ab51-fec2-44f9-b8c0-def0f46fbf3c ok Subnet:d266b4b9-62d4-4937-87e8-ca74dd2cb10f ok SecurityGroup:be39af39-b228-4126-aea7-3a63a02c882a ok Instance:0c222c03-a46e-41d3-b6d2-ec36b8c8eb91 ok Router:523fe7bf-5faf-44c4-93a0-6d437a702016 ok SecurityGroup:ff4ee3de-eb92-4724-8a88-fc8ecee20fb9 ok Router-IF:d266b4b9-62d4-4937-87e8-ca74dd2cb10f ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:a23b1bcd-9314-4af2-b993-dd7e33cf6235 ok ServerGroup:2280c0cd-7b9b-4686-9afb-e5691b20b7c4 ok Port:04242209-0aff-4ea7-b6f0-2d069b367dea ok --- PASS: TestLifecycleFloatingIPOpenstack (1.65s) === RUN TestLifecyclePrivateCalico I1206 14:25:35.228922 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:35.229194 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:35.229252 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:35.229308 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:35.229362 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:35.229406 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:35.229436 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:35.229475 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:35.229505 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:35.229573 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I1206 14:25:35.258692 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:35.262436 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.262535 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.262599 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.302160 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:35.785440 35400 iamrole.go:89] ListRoles: { } I1206 14:25:35.787832 35400 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1206 14:25:35.788066 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.788297 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.788421 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I1206 14:25:35.788508 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.788550 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1206 14:25:35.788701 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:35.788725 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:35.788642 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.788725 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I1206 14:25:35.788832 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I1206 14:25:35.788828 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:35.789109 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:35.789142 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I1206 14:25:35.789198 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:35.789236 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" }] } I1206 14:25:35.789383 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:35.789500 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:35.789540 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.789646 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.789716 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:35.789745 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:35.789794 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:35.789820 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:35.789860 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:35.789906 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.790754 35400 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1206 14:25:35.790951 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1206 14:25:35.790968 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.791160 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1206 14:25:35.791116 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:35.791210 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1206 14:25:35.791277 35400 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1206 14:25:35.791258 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1206 14:25:35.791256 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I1206 14:25:35.791438 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:35.791497 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:35.791507 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1206 14:25:35.791636 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:35.791689 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1206 14:25:35.791701 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.791767 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1206 14:25:35.791928 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:35.791986 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1206 14:25:35.792185 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.792286 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1206 14:25:35.792245 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1206 14:25:35.792391 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1206 14:25:35.792384 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:35.792582 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:35.792647 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.792743 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1206 14:25:35.792855 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1206 14:25:35.792981 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:35.793077 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1206 14:25:35.793184 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:35.793144 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:35.793228 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:35.793280 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.793031 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1206 14:25:35.793356 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:35.793442 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:35.793500 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I1206 14:25:35.793593 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:35.793659 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:35.793694 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:35.793801 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1206 14:25:35.793825 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:35.793942 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:35.793999 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:35.794052 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:35.794141 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:35.794184 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:35.794238 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:35.794304 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:35.794349 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:35.794457 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:35.794514 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:35.794602 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:35.794664 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:35.794712 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:35.796171 35400 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1206 14:25:35.796332 35400 api.go:53] PageSize not implemented W1206 14:25:35.796462 35400 api.go:53] PageSize not implemented I1206 14:25:35.796369 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:35.796646 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:35.796640 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1206 14:25:35.797175 35400 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1206 14:25:35.797209 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:35.797247 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1206 14:25:35.797284 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1206 14:25:35.797267 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.797341 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1206 14:25:35.797482 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1206 14:25:35.797497 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.797593 35400 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I1206 14:25:35.797669 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1206 14:25:35.797647 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:35.797704 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1206 14:25:35.797764 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I1206 14:25:35.797774 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:35.797848 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:35.797958 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:35.798047 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:35.798131 35400 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.798131 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1206 14:25:35.798287 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.798487 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.798628 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.798758 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I1206 14:25:35.798905 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:35.798962 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.799095 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I1206 14:25:35.799231 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.799362 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.799495 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I1206 14:25:35.799600 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.799736 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.799870 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.799984 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.800047 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.800247 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" }] }] } I1206 14:25:35.800407 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1206 14:25:35.800526 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1206 14:25:35.800579 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1206 14:25:35.800620 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.800759 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I1206 14:25:35.800891 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.801029 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I1206 14:25:35.801152 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.801281 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.801395 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.801472 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:35.801553 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1206 14:25:35.801618 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1206 14:25:35.801753 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.801898 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.802003 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.802097 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I1206 14:25:35.802298 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.806415 35400 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1206 14:25:35.806780 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:35.806813 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:35.806965 35400 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:35.807578 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:35.807641 35400 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I1206 14:25:35.807756 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:35.807834 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:35.807882 35400 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1206 14:25:35.807952 35400 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1206 14:25:35.809155 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:35.809354 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1206 14:25:35.809464 35400 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1206 14:25:35.809519 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:35.809565 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:35.809632 35400 dns.go:238] Pre-creating DNS records I1206 14:25:35.809664 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:35.817204 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I1206 14:25:35.821126 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.821220 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.821301 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:35.859339 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.322615 35400 iamrole.go:89] ListRoles: { } I1206 14:25:36.324366 35400 executor.go:111] Tasks: 0 done / 127 total; 50 can run I1206 14:25:36.324687 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.324856 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:36.324901 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:36.325021 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.324986 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:36.325099 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.325359 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.325414 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.325576 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.325767 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.328938 35400 executor.go:111] Tasks: 50 done / 127 total; 28 can run I1206 14:25:36.329095 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I1206 14:25:36.329359 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.329596 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I1206 14:25:36.329545 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.329734 35400 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I1206 14:25:36.329979 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I1206 14:25:36.329792 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I1206 14:25:36.330731 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.330942 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:36.330992 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.332014 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:36.332093 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I1206 14:25:36.332241 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.332350 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I1206 14:25:36.332482 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.332571 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:36.332708 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I1206 14:25:36.332999 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:36.333066 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:36.333130 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:36.333214 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I1206 14:25:36.333450 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:36.337606 35400 executor.go:111] Tasks: 78 done / 127 total; 40 can run W1206 14:25:36.337850 35400 api.go:53] PageSize not implemented W1206 14:25:36.337891 35400 api.go:53] PageSize not implemented I1206 14:25:36.337925 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1206 14:25:36.337899 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.338038 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1206 14:25:36.338080 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I1206 14:25:36.338100 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.338222 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1206 14:25:36.338363 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I1206 14:25:36.338338 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.338402 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I1206 14:25:36.338664 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.339003 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.339185 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.339250 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.339337 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.339390 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.339460 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.339557 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.339683 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:36.339740 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1206 14:25:36.339782 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1206 14:25:36.342087 35400 executor.go:111] Tasks: 118 done / 127 total; 5 can run I1206 14:25:36.342188 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:36.342290 35400 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W1206 14:25:36.342300 35400 api.go:53] PageSize not implemented I1206 14:25:36.342347 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I1206 14:25:36.342356 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.342455 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W1206 14:25:36.342510 35400 api.go:53] PageSize not implemented I1206 14:25:36.342551 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1206 14:25:36.342767 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I1206 14:25:36.342904 35400 executor.go:111] Tasks: 123 done / 127 total; 4 can run I1206 14:25:36.343106 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.343319 35400 executor.go:111] Tasks: 127 done / 127 total; 0 can run I1206 14:25:36.347670 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:36.347733 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:36.347773 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:36.347808 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:36.347843 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:36.347883 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1206 14:25:36.347918 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:36.347952 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:36.347984 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:36.348018 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:36.348051 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:36.348088 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:36.348128 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:36.348167 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1206 14:25:36.348205 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:36.348242 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:36.348292 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:36.348328 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:36.348367 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:36.348400 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:36.348435 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:36.348472 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:36.348510 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:36.348546 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:36.348581 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:36.348616 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:36.348654 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:36.348686 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:36.348722 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:36.348754 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:36.348785 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:36.348839 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:36.348878 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:36.348913 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:36.350893 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:36.350923 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:36.350964 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351028 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1206 14:25:36.351106 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351168 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351220 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351288 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351331 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351376 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I1206 14:25:36.351443 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351500 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:36.351527 35400 natgateway.go:117] DescribeNatGateways: { } I1206 14:25:36.351563 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I1206 14:25:36.351620 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W1206 14:25:36.351678 35400 api.go:53] PageSize not implemented I1206 14:25:36.351695 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I1206 14:25:36.351747 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:36.351775 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:36.351799 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:36.351823 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:36.351843 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:36.351898 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.351934 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:36.351980 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:36.352025 35400 iamrole.go:89] ListRoles: { } I1206 14:25:36.352075 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:36.352119 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:36.352159 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1206 14:25:36.352196 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:36.352232 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1206 14:25:36.352875 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok autoscaling-group:bastion.privatecalico.example.com ok I1206 14:25:36.352926 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:36.352972 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1206 14:25:36.353030 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I1206 14:25:36.352997 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I1206 14:25:36.353015 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I1206 14:25:36.353081 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1206 14:25:36.353133 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I1206 14:25:36.353122 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:36.353165 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1206 14:25:36.353209 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } autoscaling-group:nodes.privatecalico.example.com ok iam-instance-profile:nodes.privatecalico.example.com ok autoscaling-config:lt-2 ok volume:vol-1 ok I1206 14:25:36.353136 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I1206 14:25:36.353214 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W1206 14:25:36.353471 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.353497 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-3 ok I1206 14:25:36.353569 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:36.353667 35400 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1206 14:25:36.353737 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:36.353806 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:36.353841 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:36.353902 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1206 14:25:36.353996 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.354013 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1206 14:25:36.354057 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:36.354153 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1206 14:25:36.354288 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-1 ok I1206 14:25:36.354330 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1206 14:25:36.354423 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I1206 14:25:36.354460 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:36.354486 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I1206 14:25:36.354573 35400 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I1206 14:25:36.354591 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I1206 14:25:36.354631 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I1206 14:25:36.354668 35400 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I1206 14:25:36.354681 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I1206 14:25:36.354723 35400 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I1206 14:25:36.354561 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } I1206 14:25:36.354755 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } W1206 14:25:36.354786 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.354804 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I1206 14:25:36.354816 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I1206 14:25:36.354822 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:36.354849 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I1206 14:25:36.354872 35400 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1206 14:25:36.355009 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:36.355076 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1206 14:25:36.355131 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:36.355184 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:36.355294 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.355307 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1206 14:25:36.355375 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1206 14:25:36.355427 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:36.355551 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:36.355634 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:36.355946 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (1.13s) === RUN TestLifecyclePrivateKopeio I1206 14:25:36.356220 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:36.356497 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:36.356547 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:36.356591 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:36.356656 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:36.356697 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:36.356728 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:36.356763 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:36.356791 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:36.356855 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I1206 14:25:36.386412 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:36.389879 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.389974 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.390039 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.428913 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.548489 35400 iamrole.go:89] ListRoles: { } I1206 14:25:36.551031 35400 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1206 14:25:36.551362 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I1206 14:25:36.551526 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1206 14:25:36.551584 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:36.551696 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.551865 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.551925 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.551990 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I1206 14:25:36.552090 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:36.552158 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:36.552213 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I1206 14:25:36.552222 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.552345 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1206 14:25:36.552400 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:36.552445 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1206 14:25:36.552456 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.552675 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:36.552823 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:36.552878 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.553057 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:36.553186 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:36.553234 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:36.553278 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.553312 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.553362 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.553507 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.554462 35400 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1206 14:25:36.554735 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1206 14:25:36.554821 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1206 14:25:36.554897 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1206 14:25:36.554600 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1206 14:25:36.555068 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.555137 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1206 14:25:36.556058 35400 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1206 14:25:36.556179 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.556349 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:36.556191 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1206 14:25:36.556470 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:36.556498 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1206 14:25:36.556601 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:36.556665 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1206 14:25:36.556669 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1206 14:25:36.556912 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:36.557191 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1206 14:25:36.557410 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1206 14:25:36.557634 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.559545 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.559653 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1206 14:25:36.560428 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.559774 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:36.560603 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.560710 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.560748 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1206 14:25:36.560810 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.560860 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I1206 14:25:36.560908 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1206 14:25:36.561041 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.561128 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.561230 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I1206 14:25:36.561319 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:36.561362 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1206 14:25:36.561379 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:36.561417 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:36.561546 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:36.561654 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:36.561552 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I1206 14:25:36.561726 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I1206 14:25:36.561791 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:36.561887 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.561974 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:36.561903 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I1206 14:25:36.562041 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1206 14:25:36.561626 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1206 14:25:36.562094 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:36.562137 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1206 14:25:36.562179 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:36.562268 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:36.562318 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:36.562370 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I1206 14:25:36.562454 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:36.562504 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:36.562558 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:36.562615 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:36.562662 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I1206 14:25:36.562765 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:36.562817 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:36.562878 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:36.562921 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:36.562967 35400 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I1206 14:25:36.563052 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:36.563110 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:36.563149 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:36.563201 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1206 14:25:36.563253 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1206 14:25:36.565894 35400 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1206 14:25:36.566018 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.566204 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } W1206 14:25:36.566262 35400 api.go:53] PageSize not implemented I1206 14:25:36.566265 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1206 14:25:36.566321 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W1206 14:25:36.566398 35400 api.go:53] PageSize not implemented I1206 14:25:36.566529 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:36.566595 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1206 14:25:36.566584 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:36.566690 35400 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1206 14:25:36.566765 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1206 14:25:36.566801 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1206 14:25:36.566839 35400 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1206 14:25:36.566908 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1206 14:25:36.566900 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1206 14:25:36.566960 35400 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1206 14:25:36.567016 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1206 14:25:36.566993 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.567065 35400 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I1206 14:25:36.567107 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1206 14:25:36.567108 35400 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I1206 14:25:36.567195 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.567235 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1206 14:25:36.567258 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I1206 14:25:36.567293 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1206 14:25:36.567310 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.567378 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1206 14:25:36.567490 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.567645 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.567750 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.567952 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.568075 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.568186 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1206 14:25:36.568313 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:36.568425 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1206 14:25:36.568780 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.568914 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:36.569064 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I1206 14:25:36.569191 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.569329 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I1206 14:25:36.569369 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:36.569414 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" }] }] } I1206 14:25:36.569600 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.569658 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.569804 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.569936 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.570053 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.570187 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.570323 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.570455 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.570537 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.570671 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1206 14:25:36.570854 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1206 14:25:36.570922 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.571159 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I1206 14:25:36.571328 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:36.571436 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:36.571572 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I1206 14:25:36.571749 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:36.571794 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1206 14:25:36.571832 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:36.571965 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.572048 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.572094 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I1206 14:25:36.572134 35400 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I1206 14:25:36.572183 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:36.572225 35400 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1206 14:25:36.572262 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:36.572303 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I1206 14:25:36.572346 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1206 14:25:36.572381 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I1206 14:25:36.572512 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.573552 35400 executor.go:111] Tasks: 117 done / 126 total; 6 can run I1206 14:25:36.573624 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.573766 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I1206 14:25:36.573766 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:36.573831 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1206 14:25:36.574035 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-3" } I1206 14:25:36.574035 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:36.574255 35400 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1206 14:25:36.574494 35400 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1206 14:25:36.574585 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.574646 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:36.574739 35400 dns.go:238] Pre-creating DNS records I1206 14:25:36.574789 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:36.581423 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:36.584801 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.584897 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.584984 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.625279 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.749259 35400 iamrole.go:89] ListRoles: { } I1206 14:25:36.751732 35400 executor.go:111] Tasks: 0 done / 126 total; 47 can run I1206 14:25:36.751954 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:36.752514 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I1206 14:25:36.752739 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.752835 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:36.752960 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.753236 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.753439 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.753588 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:36.753630 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:36.753709 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.754790 35400 executor.go:111] Tasks: 47 done / 126 total; 31 can run I1206 14:25:36.755001 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1206 14:25:36.754929 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.755210 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.755316 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1206 14:25:36.755385 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.755464 35400 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I1206 14:25:36.755504 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] }] } I1206 14:25:36.755661 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I1206 14:25:36.755708 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.755843 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:36.755885 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.756017 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I1206 14:25:36.756179 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:36.756229 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.756426 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:36.756489 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.756652 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:36.756701 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:36.756843 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I1206 14:25:36.757007 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:36.757061 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.757199 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I1206 14:25:36.757345 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:36.757400 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I1206 14:25:36.757647 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:36.757728 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I1206 14:25:36.757936 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:36.760178 35400 executor.go:111] Tasks: 78 done / 126 total; 39 can run I1206 14:25:36.760532 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } W1206 14:25:36.760645 35400 api.go:53] PageSize not implemented I1206 14:25:36.760650 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.760688 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } W1206 14:25:36.760809 35400 api.go:53] PageSize not implemented I1206 14:25:36.760789 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.760879 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1206 14:25:36.760855 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.761039 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.761103 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I1206 14:25:36.761022 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.761205 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I1206 14:25:36.761247 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I1206 14:25:36.761339 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.761448 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:36.761537 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1206 14:25:36.761719 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.761936 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.762098 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.762218 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.762473 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.762660 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:36.764515 35400 executor.go:111] Tasks: 117 done / 126 total; 6 can run W1206 14:25:36.764633 35400 api.go:53] PageSize not implemented I1206 14:25:36.764610 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I1206 14:25:36.764661 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.764700 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:36.764811 35400 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" W1206 14:25:36.764764 35400 api.go:53] PageSize not implemented I1206 14:25:36.764759 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:36.764865 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.765071 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I1206 14:25:36.765173 35400 executor.go:111] Tasks: 123 done / 126 total; 3 can run I1206 14:25:36.765315 35400 executor.go:111] Tasks: 126 done / 126 total; 0 can run I1206 14:25:36.768941 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:36.769007 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:36.769049 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:36.769086 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:36.769118 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:36.769156 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:36.769189 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:36.769224 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:36.769259 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:36.769295 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:36.769333 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:36.769372 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:36.769414 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1206 14:25:36.769454 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:36.769489 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:36.769530 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:36.769569 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:36.769605 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:36.769644 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I1206 14:25:36.769681 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:36.769717 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:36.769753 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:36.769792 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:36.769828 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:36.769863 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:36.769900 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:36.769935 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:36.769973 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1206 14:25:36.770013 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I1206 14:25:36.770050 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:36.770085 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:36.770119 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:36.770156 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:36.770192 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:36.770230 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:36.772000 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:36.772028 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:36.772067 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772132 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1206 14:25:36.772213 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772272 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772320 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772370 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772410 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772461 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I1206 14:25:36.772529 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772600 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:36.772633 35400 natgateway.go:117] DescribeNatGateways: { } I1206 14:25:36.772680 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I1206 14:25:36.772724 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W1206 14:25:36.772777 35400 api.go:53] PageSize not implemented I1206 14:25:36.772794 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I1206 14:25:36.772847 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:36.772874 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:36.772891 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:36.772917 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:36.772934 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:36.772984 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:36.773012 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:36.773059 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:36.773096 35400 iamrole.go:89] ListRoles: { } I1206 14:25:36.773137 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:36.773184 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:36.773232 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-3","rtb-1"] } I1206 14:25:36.773275 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I1206 14:25:36.773306 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I1206 14:25:36.773939 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.privatekopeio.example.com ok I1206 14:25:36.773998 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I1206 14:25:36.774081 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } autoscaling-group:bastion.privatekopeio.example.com ok I1206 14:25:36.774011 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } iam-instance-profile:nodes.privatekopeio.example.com ok I1206 14:25:36.774100 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W1206 14:25:36.774147 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.774142 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1206 14:25:36.774183 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:36.774199 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I1206 14:25:36.774192 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I1206 14:25:36.774233 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1206 14:25:36.774221 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok load-balancer:bastion-privatekopeio-exa-d8ef8e ok I1206 14:25:36.774278 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I1206 14:25:36.774367 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:36.774469 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:36.774296 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.privatekopeio.example.com. ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I1206 14:25:36.774519 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:36.774581 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W1206 14:25:36.774689 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.774706 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1206 14:25:36.774742 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:36.774797 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W1206 14:25:36.775014 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.775036 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:36.775071 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:36.775144 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:36.775287 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1206 14:25:36.775304 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I1206 14:25:36.775354 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:36.775406 35400 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I1206 14:25:36.775501 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I1206 14:25:36.775559 35400 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I1206 14:25:36.775600 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I1206 14:25:36.775657 35400 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I1206 14:25:36.775670 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I1206 14:25:36.775699 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I1206 14:25:36.775731 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I1206 14:25:36.776250 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I1206 14:25:36.776287 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I1206 14:25:36.776345 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I1206 14:25:36.775448 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:36.776422 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.776452 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1206 14:25:36.776520 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I1206 14:25:36.776631 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:36.776700 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1206 14:25:36.776855 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:36.776873 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:36.776921 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1206 14:25:36.776971 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1206 14:25:36.777063 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I1206 14:25:36.777150 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:36.777292 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1206 14:25:36.777472 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:36.777612 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:36.778157 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.42s) === RUN TestLifecycleIPv6 I1206 14:25:36.778707 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:36.779323 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:36.779384 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:36.779442 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:36.779504 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:36.779548 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:36.779577 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:36.779615 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:36.780410 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:36.780500 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I1206 14:25:36.807764 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:36.811299 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.811399 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:36.850830 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.027598 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.030952 35400 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1206 14:25:37.031552 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.031798 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.031826 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.031802 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:37.032070 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1206 14:25:37.032145 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.032348 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.032531 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:37.032612 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.032716 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.033520 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.033606 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.033813 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.032843 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I1206 14:25:37.034001 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.034947 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.035195 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.035431 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:37.035524 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:37.035608 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:37.035666 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:37.035716 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:37.035791 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.036743 35400 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1206 14:25:37.036869 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.037140 35400 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1206 14:25:37.037029 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1206 14:25:37.037187 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1206 14:25:37.037312 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1206 14:25:37.037288 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.037433 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I1206 14:25:37.037311 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.037508 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.037518 35400 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1206 14:25:37.037578 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:37.037622 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:37.037706 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.037784 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:37.037873 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.037923 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:37.037969 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.038123 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:37.038173 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1206 14:25:37.038305 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.038423 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1206 14:25:37.038597 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.038774 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:37.038902 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:37.038967 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.039042 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.039195 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:37.039282 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:37.039409 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:37.039479 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:37.039551 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:37.039723 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1206 14:25:37.039785 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:37.039874 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" } ] } I1206 14:25:37.039966 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:37.039653 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.040030 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:37.043580 35400 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1206 14:25:37.043706 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.043856 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.044012 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.044210 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:37.044358 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:37.044594 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:37.044805 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.044973 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1206 14:25:37.045085 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1206 14:25:37.045400 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.045533 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.045616 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.045729 35400 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.045759 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1206 14:25:37.045951 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.046216 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:37.046299 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.046417 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.046521 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" }] }] } I1206 14:25:37.047120 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.047221 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" }] }] } I1206 14:25:37.047454 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:37.047409 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:37.047520 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.047719 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I1206 14:25:37.047861 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.047912 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1206 14:25:37.048162 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.048455 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1206 14:25:37.048625 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.048703 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:37.048772 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.049001 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.049246 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I1206 14:25:37.049482 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.049717 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" }] }] } I1206 14:25:37.049989 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.050080 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:37.051911 35400 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1206 14:25:37.052043 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:37.052134 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:37.052078 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.052265 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:37.052323 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:37.052267 35400 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I1206 14:25:37.052364 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.052430 35400 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.052411 35400 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I1206 14:25:37.052520 35400 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1206 14:25:37.052557 35400 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I1206 14:25:37.052667 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:37.052714 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.052755 35400 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1206 14:25:37.052835 35400 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1206 14:25:37.052954 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.052949 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.053101 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.053142 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I1206 14:25:37.053207 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.053327 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.053434 35400 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1206 14:25:37.053484 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.053514 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.053559 35400 dns.go:238] Pre-creating DNS records I1206 14:25:37.053583 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.059438 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:37.062907 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.063004 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.105537 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.281687 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.283566 35400 executor.go:111] Tasks: 0 done / 106 total; 45 can run I1206 14:25:37.283963 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.284104 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:37.284159 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:37.284219 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.284263 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.284374 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] }] } I1206 14:25:37.284420 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:37.284535 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.284648 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.284825 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.286898 35400 executor.go:111] Tasks: 45 done / 106 total; 21 can run I1206 14:25:37.287046 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1206 14:25:37.287044 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I1206 14:25:37.287270 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.287403 35400 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1206 14:25:37.287394 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:37.287447 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I1206 14:25:37.287537 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1206 14:25:37.287516 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I1206 14:25:37.287754 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.287863 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.287996 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.288119 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I1206 14:25:37.288242 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.288322 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.288452 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.292357 35400 executor.go:111] Tasks: 66 done / 106 total; 30 can run I1206 14:25:37.292529 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.292682 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.292802 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.293058 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:37.293195 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.293339 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.293428 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:37.293470 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I1206 14:25:37.293674 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.293985 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.294148 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.294321 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.294457 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.295617 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1206 14:25:37.301643 35400 executor.go:111] Tasks: 96 done / 106 total; 5 can run I1206 14:25:37.301807 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.301935 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.301933 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:37.302030 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:37.302019 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:37.302052 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1206 14:25:37.302126 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1206 14:25:37.302166 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1206 14:25:37.302214 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I1206 14:25:37.302252 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I1206 14:25:37.302398 35400 executor.go:111] Tasks: 101 done / 106 total; 5 can run I1206 14:25:37.302484 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.302582 35400 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1206 14:25:37.302611 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:37.302635 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1206 14:25:37.302560 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.302662 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.302765 35400 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I1206 14:25:37.302790 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:37.302815 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1206 14:25:37.302908 35400 executor.go:111] Tasks: 106 done / 106 total; 0 can run I1206 14:25:37.306423 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:37.306487 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:37.306524 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:37.306559 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:37.306592 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:37.306629 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1206 14:25:37.306664 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:37.306696 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:37.306729 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:37.306761 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:37.306795 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:37.306829 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:37.306866 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:37.306907 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:37.306942 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:37.306997 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:37.307055 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:37.307121 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:37.307166 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:37.307220 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:37.307273 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:37.307331 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:37.307367 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.307410 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:37.307445 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:37.307477 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:37.307509 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:37.307548 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:37.307584 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:37.307621 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:37.309188 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:37.309219 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:37.309262 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309318 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1206 14:25:37.309380 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309441 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309495 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309547 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309588 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309633 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I1206 14:25:37.309686 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309735 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:37.309762 35400 natgateway.go:117] DescribeNatGateways: { } I1206 14:25:37.309804 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I1206 14:25:37.309846 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W1206 14:25:37.309924 35400 api.go:53] PageSize not implemented I1206 14:25:37.309943 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:37.309968 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:37.309982 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I1206 14:25:37.310019 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:37.310042 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:37.310058 35400 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I1206 14:25:37.310117 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.310144 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.310189 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:37.310226 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.310272 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:37.310322 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:37.310364 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1206 14:25:37.310400 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:37.310434 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.minimal-ipv6.example.com ok I1206 14:25:37.310969 35400 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I1206 14:25:37.311055 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I1206 14:25:37.311112 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.311037 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } iam-instance-profile:nodes.minimal-ipv6.example.com ok volume:vol-1 ok I1206 14:25:37.311172 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.311256 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:37.311294 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:37.311290 35400 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } autoscaling-config:lt-2 ok load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok I1206 14:25:37.311385 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:37.311386 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.311450 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I1206 14:25:37.311453 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:37.311519 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1206 14:25:37.311594 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:37.311679 35400 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1206 14:25:37.311477 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.311739 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I1206 14:25:37.311794 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1206 14:25:37.311958 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:37.311981 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:37.312015 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:37.312070 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W1206 14:25:37.312344 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:37.312364 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-1 ok I1206 14:25:37.312449 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.312504 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I1206 14:25:37.312520 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I1206 14:25:37.312488 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:37.312554 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I1206 14:25:37.312586 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.312582 35400 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I1206 14:25:37.312629 35400 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I1206 14:25:37.312641 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I1206 14:25:37.312667 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I1206 14:25:37.312726 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:37.312804 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:37.312892 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:37.313181 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.53s) === RUN TestLifecycleSharedVPC I1206 14:25:37.313454 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:37.313742 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:37.313789 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:37.313834 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:37.313881 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:37.313923 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:37.313952 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:37.313993 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:37.314023 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:37.314085 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I1206 14:25:37.338378 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:37.341414 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.341509 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.380842 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.486045 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.487608 35400 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1206 14:25:37.487969 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I1206 14:25:37.488088 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I1206 14:25:37.488252 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I1206 14:25:37.488356 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I1206 14:25:37.488338 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.488453 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:37.488525 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:37.488640 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:37.488694 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:37.488765 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.488857 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.488932 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.489160 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:37.489232 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] }] } I1206 14:25:37.489475 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.489643 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:37.489709 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1206 14:25:37.490556 35400 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1206 14:25:37.490715 35400 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1206 14:25:37.490688 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.490919 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1206 14:25:37.490833 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1206 14:25:37.490973 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1206 14:25:37.490912 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I1206 14:25:37.491111 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1206 14:25:37.491121 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:37.491285 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.491392 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:37.491522 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], VpcId: "vpc-12345678" } I1206 14:25:37.491502 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1206 14:25:37.491537 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1206 14:25:37.491811 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I1206 14:25:37.491928 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:37.491980 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.492037 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.492091 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:37.492141 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:37.492266 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:37.492332 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.492376 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:37.492423 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:37.494039 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1206 14:25:37.497412 35400 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1206 14:25:37.497640 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:37.497830 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.497981 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" }] }] } I1206 14:25:37.498198 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.498372 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.498507 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1206 14:25:37.498738 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.498858 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.498929 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.498997 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.499072 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.499205 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.499267 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.499406 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.499481 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.499713 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1206 14:25:37.499873 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" }] }] } I1206 14:25:37.500027 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.500183 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1206 14:25:37.500239 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I1206 14:25:37.500371 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1206 14:25:37.500421 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:37.500474 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:37.500514 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.500652 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I1206 14:25:37.501499 35400 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1206 14:25:37.501796 35400 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1206 14:25:37.501936 35400 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1206 14:25:37.501982 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.502021 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.502078 35400 dns.go:238] Pre-creating DNS records I1206 14:25:37.502107 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.507620 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:37.510653 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.510747 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.549275 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.656177 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.657892 35400 executor.go:111] Tasks: 0 done / 84 total; 42 can run I1206 14:25:37.658312 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:37.658430 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.658556 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:37.658676 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.658822 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:37.659072 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:37.659139 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:37.659230 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.661122 35400 executor.go:111] Tasks: 42 done / 84 total; 17 can run I1206 14:25:37.661265 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I1206 14:25:37.661449 35400 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I1206 14:25:37.661629 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1206 14:25:37.661518 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.661736 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.661805 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:37.661932 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I1206 14:25:37.662102 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.662161 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] }] } I1206 14:25:37.662303 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:37.670245 35400 executor.go:111] Tasks: 59 done / 84 total; 21 can run I1206 14:25:37.670509 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.670761 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.670973 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.671282 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.671441 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.671506 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.672154 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.673535 35400 executor.go:111] Tasks: 80 done / 84 total; 2 can run I1206 14:25:37.673758 35400 executor.go:111] Tasks: 82 done / 84 total; 2 can run I1206 14:25:37.673892 35400 executor.go:111] Tasks: 84 done / 84 total; 0 can run I1206 14:25:37.676703 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:37.676768 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:37.676804 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:37.676835 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:37.676868 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:37.676902 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:37.676938 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:37.676972 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:37.677007 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:37.677044 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:37.677082 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:37.677114 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:37.677142 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:37.677179 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:37.677212 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:37.677247 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:37.677285 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.677323 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:37.677355 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:37.677387 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:37.677418 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:37.677455 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:37.677493 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:37.678885 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:37.678918 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:37.678957 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679009 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1206 14:25:37.679066 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679123 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679166 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679207 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679247 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679288 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I1206 14:25:37.679336 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679393 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I1206 14:25:37.679429 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W1206 14:25:37.679476 35400 api.go:53] PageSize not implemented I1206 14:25:37.679498 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:37.679525 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:37.679545 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:37.679570 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:37.679586 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:37.679637 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.679667 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.679709 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:37.679749 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.679792 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:37.679830 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:37.679859 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I1206 14:25:37.680299 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } autoscaling-config:lt-1 ok I1206 14:25:37.680392 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I1206 14:25:37.680250 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.sharedvpc.example.com ok I1206 14:25:37.680397 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok route53-record:Z1AFAKE1ZON3YO/A/api.sharedvpc.example.com. ok autoscaling-config:lt-2 ok I1206 14:25:37.680478 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.680495 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1206 14:25:37.680542 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I1206 14:25:37.680540 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } W1206 14:25:37.680753 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:37.680776 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:37.680812 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1206 14:25:37.680871 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:37.680926 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.680973 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1206 14:25:37.681083 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:37.681096 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:37.681173 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:37.681222 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I1206 14:25:37.681356 35400 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I1206 14:25:37.681391 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I1206 14:25:37.681467 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I1206 14:25:37.681528 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I1206 14:25:37.681621 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I1206 14:25:37.681657 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I1206 14:25:37.681707 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I1206 14:25:37.681794 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:37.682178 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.37s) === RUN TestLifecycleComplex I1206 14:25:37.682472 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:37.682727 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:37.682774 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:37.682819 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:37.682865 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:37.682905 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:37.682935 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:37.682972 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:37.683000 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:37.683063 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I1206 14:25:37.718606 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:37.722793 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.722887 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.767028 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.971229 35400 iamrole.go:89] ListRoles: { } I1206 14:25:37.973542 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1206 14:25:37.973886 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:37.974035 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.974169 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I1206 14:25:37.974233 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.974289 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } W1206 14:25:37.974472 35400 api.go:53] PageSize not implemented I1206 14:25:37.974388 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I1206 14:25:37.974503 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I1206 14:25:37.974606 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:37.974633 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I1206 14:25:37.974607 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:37.974797 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:37.974963 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:37.975049 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:37.975270 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:37.975264 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.975392 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:37.975427 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:37.975465 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:37.975509 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.975563 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.975745 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.975910 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:37.976043 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:37.977239 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:37.978821 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:37.979088 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:37.979008 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.979226 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:37.979337 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:37.979243 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.979368 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:37.979421 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:37.979560 35400 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1206 14:25:37.979604 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.979501 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1206 14:25:37.980107 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1206 14:25:37.980234 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.980303 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1206 14:25:37.980548 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1206 14:25:37.980543 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.981382 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.980342 35400 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I1206 14:25:37.981567 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.981745 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:37.981826 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1206 14:25:37.981947 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:37.981869 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1206 14:25:37.982832 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:37.982984 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.982953 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I1206 14:25:37.983090 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:37.983192 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:37.983332 35400 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I1206 14:25:37.983406 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.983545 35400 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I1206 14:25:37.983589 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:37.983669 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1206 14:25:37.983816 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:37.983798 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1206 14:25:37.983880 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:37.984001 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:37.984064 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:37.984124 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1206 14:25:37.984173 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:37.984212 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I1206 14:25:37.984329 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:37.984378 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:37.984409 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:37.984517 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:37.984570 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:37.984613 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:37.984736 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:37.984803 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:37.984880 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:37.984984 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:37.985153 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:37.985247 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I1206 14:25:37.985405 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:37.985495 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:37.985248 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1206 14:25:37.987204 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:37.987365 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.987466 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.987573 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.987556 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:37.987780 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:37.987779 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I1206 14:25:37.987858 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I1206 14:25:37.987980 35400 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ], Type: "network" } I1206 14:25:37.988032 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1206 14:25:37.988226 35400 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1206 14:25:37.988348 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I1206 14:25:37.988429 35400 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1206 14:25:37.988469 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.988552 35400 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:37.988655 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:37.988792 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.988928 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:37.989063 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.989181 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1206 14:25:37.989486 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1206 14:25:37.989669 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I1206 14:25:37.989725 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:37.989808 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1206 14:25:37.989953 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I1206 14:25:37.990054 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.990214 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.990265 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I1206 14:25:37.990366 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.990446 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I1206 14:25:37.990548 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1206 14:25:37.990697 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1206 14:25:37.990781 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" } ] }] } I1206 14:25:37.990921 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I1206 14:25:37.990962 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.991112 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1206 14:25:37.991151 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" } ] }] } I1206 14:25:37.991334 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I1206 14:25:37.991424 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1206 14:25:37.991582 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1206 14:25:37.991858 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I1206 14:25:37.991942 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.992195 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I1206 14:25:37.992442 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I1206 14:25:37.992668 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I1206 14:25:37.992755 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I1206 14:25:37.992897 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1206 14:25:37.992985 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1206 14:25:37.993131 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I1206 14:25:37.993219 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I1206 14:25:37.993306 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:37.993350 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:37.993413 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1206 14:25:37.993479 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I1206 14:25:37.993637 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I1206 14:25:37.996911 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:37.997032 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.997201 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:37.997340 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:37.997478 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1206 14:25:37.997524 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:37.997557 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:37.997606 35400 dns.go:238] Pre-creating DNS records I1206 14:25:37.997631 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:38.005887 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:38.010153 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.010257 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.049483 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.247484 35400 iamrole.go:89] ListRoles: { } I1206 14:25:38.249255 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1206 14:25:38.249497 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W1206 14:25:38.249647 35400 api.go:53] PageSize not implemented I1206 14:25:38.249660 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:38.249756 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:38.249904 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:38.250141 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.252400 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.252666 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:38.252838 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.253118 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:38.253340 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:38.253509 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:38.253635 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.257484 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:38.257649 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:38.257761 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.257669 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.257880 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.257911 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:38.258033 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.258002 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:38.258046 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:38.258169 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:38.258216 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:38.258301 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.258524 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:38.258596 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:38.258790 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:38.258978 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:38.259059 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:38.259276 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:38.259366 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.259550 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.259659 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.259747 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.259835 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.259972 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:38.260013 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I1206 14:25:38.260138 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:38.263220 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:38.263468 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.263566 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:38.263653 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:38.263675 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.263723 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.263591 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.263759 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.263801 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.263836 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.263866 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.263973 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.264088 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.264222 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.264329 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.264494 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.264707 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.265135 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.265306 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.265573 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.268399 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:38.268578 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:38.268676 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:38.268701 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:38.268724 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.268813 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:38.268935 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I1206 14:25:38.275298 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.278949 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.291063 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:38.295227 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.295325 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.334365 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.532058 35400 iamrole.go:89] ListRoles: { } I1206 14:25:38.534349 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run W1206 14:25:38.534563 35400 api.go:53] PageSize not implemented I1206 14:25:38.534782 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.534982 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:38.535123 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:38.535067 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:38.535234 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.535395 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:38.535431 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:38.535484 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:38.535711 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.535973 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.536208 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:38.536385 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.537700 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:38.537866 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.538092 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:38.538068 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:38.538030 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:38.538168 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.538129 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:38.538232 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:38.538381 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:38.538447 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.538501 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.538556 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I1206 14:25:38.538720 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:38.538763 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:38.538922 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:38.538975 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.539127 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:38.539172 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.539296 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.539421 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.539510 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:38.539655 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:38.539730 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.540029 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:38.540116 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.540242 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:38.543354 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:38.543510 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.543599 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.543631 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:38.543748 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:38.543776 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.543726 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.543839 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.543896 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.543954 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.543964 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.544111 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.544164 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.544444 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I1206 14:25:38.544513 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.544388 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.544564 35400 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I1206 14:25:38.544691 35400 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I1206 14:25:38.544756 35400 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I1206 14:25:38.544863 35400 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I1206 14:25:38.544770 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.544958 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.545001 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.545052 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.545613 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.545792 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.545935 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.546076 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.547639 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:38.547737 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:38.547843 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:38.547868 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:38.547894 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.548014 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:38.548243 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run I1206 14:25:38.548318 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.548365 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:38.548420 35400 dns.go:238] Pre-creating DNS records I1206 14:25:38.556164 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:38.562161 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.562362 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.601557 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.794720 35400 iamrole.go:89] ListRoles: { } I1206 14:25:38.796547 35400 executor.go:111] Tasks: 0 done / 122 total; 51 can run I1206 14:25:38.797164 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:38.797370 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W1206 14:25:38.797519 35400 api.go:53] PageSize not implemented I1206 14:25:38.799377 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:38.799599 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.799790 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:38.799908 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.800072 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:38.800127 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:38.800206 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.800467 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:38.800794 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.801200 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1206 14:25:38.804889 35400 executor.go:111] Tasks: 51 done / 122 total; 27 can run I1206 14:25:38.805025 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:38.805135 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:38.805234 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.805153 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:38.805356 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:38.805334 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.805430 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:38.805530 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.805619 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:38.805709 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:38.806005 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:38.806090 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.806339 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:38.806414 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.806544 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.806747 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:38.806958 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.807077 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.807287 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.807444 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:38.807502 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.807669 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.807853 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:38.808013 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:38.808233 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:38.811444 35400 executor.go:111] Tasks: 78 done / 122 total; 39 can run I1206 14:25:38.811639 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.811830 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:38.811921 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:38.811941 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.811911 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.811991 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.812121 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.812090 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.812168 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:38.812202 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:38.812235 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:38.812274 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.812406 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.812743 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.813038 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.813140 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:38.813321 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.813542 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.813768 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:38.815238 35400 executor.go:111] Tasks: 117 done / 122 total; 3 can run I1206 14:25:38.815325 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:38.815413 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:38.815439 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:38.815465 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:38.815605 35400 executor.go:111] Tasks: 120 done / 122 total; 2 can run I1206 14:25:38.815760 35400 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I1206 14:25:38.819600 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:38.823255 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:38.835508 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:38.839765 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.839864 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:38.878533 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.080212 35400 iamrole.go:89] ListRoles: { } I1206 14:25:39.082415 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1206 14:25:39.082784 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:39.082951 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:39.083056 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W1206 14:25:39.083130 35400 api.go:53] PageSize not implemented I1206 14:25:39.083150 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.083134 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:39.083308 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:39.083448 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.083582 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:39.083615 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:39.083674 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.083718 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.083862 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:39.083973 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.085632 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:39.085783 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:39.086047 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:39.086048 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.086152 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:39.086155 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.086179 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:39.086211 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:39.086327 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.086214 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.086479 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:39.086546 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.086644 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:39.086699 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.086841 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.086932 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.087017 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.087100 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.087255 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:39.087305 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:39.087354 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.087452 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:39.087577 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.087668 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:39.087854 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:39.087942 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:39.091092 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:39.091401 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.091444 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:39.091538 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:39.091565 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.091595 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.091643 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.091768 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.091763 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.091833 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.091968 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.091967 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.092005 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.092326 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.093273 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.093577 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.093803 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.094083 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.094267 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.094435 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.099155 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.099250 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.099449 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:39.099531 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:39.099616 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:39.099644 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:39.099673 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.099761 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:39.099892 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:39.099940 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.099971 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:39.100020 35400 dns.go:238] Pre-creating DNS records I1206 14:25:39.108993 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:39.114013 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.114116 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.153340 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.356041 35400 iamrole.go:89] ListRoles: { } I1206 14:25:39.358426 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1206 14:25:39.358640 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W1206 14:25:39.358918 35400 api.go:53] PageSize not implemented I1206 14:25:39.359036 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:39.359096 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1206 14:25:39.359235 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:39.359464 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:39.359641 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.359887 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.360073 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:39.360126 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:39.360221 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.360284 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I1206 14:25:39.360529 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.360746 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:39.361340 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:39.361459 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.361667 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:39.361730 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:39.361597 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.361692 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:39.361850 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.361886 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:39.361974 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:39.361955 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.362061 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.362122 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:39.362177 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.362335 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.362463 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.362643 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.362835 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:39.362886 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.363057 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:39.363251 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:39.363299 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:39.363452 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.363544 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.363638 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:39.363826 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:39.363890 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:39.366582 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:39.366738 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.366903 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:39.366989 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:39.366990 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.367016 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.367091 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.367138 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.367196 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.367238 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.367274 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.367304 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.367278 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.367633 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.367800 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.367952 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.368040 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.368142 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.368253 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.368418 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.370986 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:39.371117 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:39.371216 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:39.371241 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:39.371264 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.371358 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:39.371491 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I1206 14:25:39.376667 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.380222 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.380312 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.389821 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:39.394045 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.394141 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.394220 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.433008 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.627337 35400 iamrole.go:89] ListRoles: { } I1206 14:25:39.629491 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1206 14:25:39.629733 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W1206 14:25:39.629945 35400 api.go:53] PageSize not implemented I1206 14:25:39.629916 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:39.630093 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:39.630048 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.630333 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:39.630551 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.630698 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:39.630734 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:39.630834 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.630895 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1206 14:25:39.631024 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.631168 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:39.631276 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.633969 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:39.634194 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:39.634390 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:39.634354 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:39.634500 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:39.634551 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.634412 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.634644 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:39.634784 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.634756 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.634990 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.635123 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:39.635334 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.635869 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:39.636533 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:39.637226 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.637378 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.637480 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.637649 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.637749 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.637902 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:39.637950 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:39.638007 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:39.638141 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:39.638194 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:39.638319 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:39.642567 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:39.642731 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.642834 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.642952 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:39.643029 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:39.643049 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.643115 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.643009 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.643162 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.643285 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.643324 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.643354 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.643625 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.643759 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.644102 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.644352 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.644553 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.644786 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.644967 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.645059 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.648151 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:39.648405 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:39.648512 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:39.648541 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:39.648568 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.648669 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:39.648830 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:39.648889 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.648925 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:39.648983 35400 dns.go:238] Pre-creating DNS records I1206 14:25:39.657011 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:39.661275 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.661377 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.661438 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.699366 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.899036 35400 iamrole.go:89] ListRoles: { } I1206 14:25:39.901476 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run I1206 14:25:39.901714 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:39.901827 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.901996 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:39.902045 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } W1206 14:25:39.902102 35400 api.go:53] PageSize not implemented I1206 14:25:39.902081 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:39.902277 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:39.902453 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.902732 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.902815 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.903004 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1206 14:25:39.903161 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.903342 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:39.903477 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:39.904138 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:39.904268 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:39.904312 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:39.904408 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.904431 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.904531 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:39.904575 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:39.904644 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:39.904588 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:39.904639 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.904717 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.904859 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:39.904979 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.905136 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.905231 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.905365 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:39.905405 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:39.905559 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:39.905616 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.905730 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:39.905802 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:39.905908 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:39.906046 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.906144 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:39.906245 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:39.906348 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:39.909537 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:39.909739 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.909868 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.910001 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:39.910091 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:39.910110 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.910156 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.910121 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.910208 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.910262 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:39.910311 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:39.910351 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:39.910350 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.910614 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:39.910783 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.910897 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:39.911279 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.911483 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.911557 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.911811 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.914582 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:39.914697 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:39.914803 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:39.914836 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:39.914867 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:39.914988 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:39.915141 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I1206 14:25:39.918294 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:39.922318 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.922409 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.922465 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.931911 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:39.940217 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.940323 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.940382 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.940432 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:39.981160 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.170395 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.172627 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1206 14:25:40.173710 35400 api.go:53] PageSize not implemented I1206 14:25:40.173085 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.174008 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:40.174191 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:40.174296 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:40.174424 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:40.174749 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:40.174928 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.175307 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.175260 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:40.175392 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:40.175596 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1206 14:25:40.175977 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.177181 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.179221 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:40.179384 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.179524 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:40.179482 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:40.179591 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:40.179552 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.179619 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:40.179707 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:40.179799 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:40.179781 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:40.179910 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.180069 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I1206 14:25:40.180219 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.180323 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:40.180527 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.180600 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.180749 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:40.180929 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:40.180973 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:40.181094 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.181209 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.181282 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.181394 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:40.181440 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.181553 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.181618 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:40.184898 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:40.185279 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.185376 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.185404 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:40.185489 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:40.185513 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.185560 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.185540 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.185599 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:40.185642 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:40.185677 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:40.185710 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:40.185720 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.185908 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.186205 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.186339 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.186516 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.187071 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.187229 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.187329 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.188677 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:40.188862 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:40.188967 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:40.188994 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:40.189020 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.189122 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:40.189355 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:40.189435 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.189516 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:40.189617 35400 dns.go:238] Pre-creating DNS records I1206 14:25:40.197809 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I1206 14:25:40.202079 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.202178 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.202243 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.202299 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.240583 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.438609 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.440412 35400 executor.go:111] Tasks: 0 done / 123 total; 52 can run W1206 14:25:40.441527 35400 api.go:53] PageSize not implemented I1206 14:25:40.441644 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I1206 14:25:40.441801 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I1206 14:25:40.442265 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.442497 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I1206 14:25:40.442582 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:40.442785 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.443035 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:40.443102 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:40.443230 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I1206 14:25:40.443495 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I1206 14:25:40.443774 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I1206 14:25:40.444075 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.444665 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.447311 35400 executor.go:111] Tasks: 52 done / 123 total; 27 can run I1206 14:25:40.447477 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.447605 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I1206 14:25:40.447694 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:40.447720 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:40.447640 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.447771 35400 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:40.447740 35400 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I1206 14:25:40.447875 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:40.447813 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.448075 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.448251 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:40.448330 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.448471 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.448637 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:40.448686 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.448781 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I1206 14:25:40.448978 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.449037 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.449115 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I1206 14:25:40.449261 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I1206 14:25:40.449384 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.449478 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.449603 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:40.449687 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.449750 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I1206 14:25:40.453743 35400 executor.go:111] Tasks: 79 done / 123 total; 39 can run I1206 14:25:40.453943 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.454095 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.454225 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:40.454313 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:40.454332 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.454377 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.454421 35400 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:40.454462 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5"] } I1206 14:25:40.454422 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.454495 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4"] } I1206 14:25:40.454584 35400 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I1206 14:25:40.454638 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.455248 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.455435 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.455698 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.455869 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:40.455986 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.456145 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.456348 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.459581 35400 executor.go:111] Tasks: 118 done / 123 total; 3 can run I1206 14:25:40.459783 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:40.459886 35400 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I1206 14:25:40.459911 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { } I1206 14:25:40.459933 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.460027 35400 executor.go:111] Tasks: 121 done / 123 total; 2 can run I1206 14:25:40.460200 35400 executor.go:111] Tasks: 123 done / 123 total; 0 can run I1206 14:25:40.460606 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:40.460685 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:40.460721 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:40.460753 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:40.460786 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:40.460829 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:40.460865 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:40.460896 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:40.460931 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:40.460969 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:40.461010 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:40.461056 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:40.461096 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:40.461130 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:40.461168 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:40.461203 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:40.461240 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:40.461278 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:40.461317 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:40.461368 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:40.461438 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:40.461498 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:40.461561 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I1206 14:25:40.461638 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:40.461679 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:40.461712 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:40.461753 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:40.461794 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:40.461831 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:40.464527 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:40.464561 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:40.464596 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.464659 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1206 14:25:40.464729 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.464791 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.464840 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.464894 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.464956 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.465008 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I1206 14:25:40.465066 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.465142 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I1206 14:25:40.465188 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W1206 14:25:40.465246 35400 api.go:53] PageSize not implemented I1206 14:25:40.465269 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:40.465299 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:40.465316 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I1206 14:25:40.465359 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:40.465384 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:40.465401 35400 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" ] } I1206 14:25:40.465469 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.465497 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:40.465539 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:40.465576 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.465618 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:40.465667 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:40.465716 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I1206 14:25:40.466287 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-config:lt-1 ok I1206 14:25:40.466353 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok autoscaling-group:nodes.complex.example.com ok I1206 14:25:40.466379 35400 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I1206 14:25:40.466405 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/5 ok I1206 14:25:40.466461 35400 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I1206 14:25:40.466501 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I1206 14:25:40.466445 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I1206 14:25:40.466540 35400 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/4 ok I1206 14:25:40.466411 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.complex.example.com. ok I1206 14:25:40.466567 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:40.466678 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W1206 14:25:40.466932 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:40.466961 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1206 14:25:40.467013 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:40.467093 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1206 14:25:40.467152 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.467225 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 } ] } W1206 14:25:40.467577 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:40.467599 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:40.467635 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1206 14:25:40.467695 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:40.467763 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:40.467797 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.467849 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I1206 14:25:40.468005 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:40.468035 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I1206 14:25:40.468118 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I1206 14:25:40.468096 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:40.468138 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I1206 14:25:40.468174 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I1206 14:25:40.468209 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I1206 14:25:40.468293 35400 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I1206 14:25:40.468319 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I1206 14:25:40.468354 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I1206 14:25:40.468430 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1206 14:25:40.468516 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:40.468605 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:40.468918 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.79s) === RUN TestLifecycleExternalLB I1206 14:25:40.469258 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:40.469519 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:40.469567 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:40.469607 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:40.469663 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:40.469708 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:40.469737 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:40.469772 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:40.469801 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:40.469866 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I1206 14:25:40.499370 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:40.502921 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.503033 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.543226 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.653370 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.655162 35400 executor.go:111] Tasks: 0 done / 94 total; 50 can run W1206 14:25:40.655510 35400 api.go:53] PageSize not implemented W1206 14:25:40.655593 35400 api.go:53] PageSize not implemented I1206 14:25:40.655604 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:40.655629 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I1206 14:25:40.655715 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:40.655744 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1206 14:25:40.655886 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I1206 14:25:40.655842 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:40.655957 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I1206 14:25:40.655911 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W1206 14:25:40.656069 35400 api.go:53] PageSize not implemented I1206 14:25:40.656059 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:40.655415 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.656189 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.656293 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.656310 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:40.656384 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.656461 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.656400 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:40.656605 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:40.656782 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:40.656837 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.657011 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1206 14:25:40.657132 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:40.657182 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.657359 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:40.657481 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.657582 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.657757 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:40.657765 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.657895 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:40.657933 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:40.657963 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:40.658011 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.658756 35400 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1206 14:25:40.659063 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1206 14:25:40.659015 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1206 14:25:40.659226 35400 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1206 14:25:40.659312 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.659430 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1206 14:25:40.659533 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:40.659481 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1206 14:25:40.659594 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.659731 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:40.659693 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1206 14:25:40.659786 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1206 14:25:40.659960 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.660048 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:40.660101 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.660202 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:40.660319 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1206 14:25:40.660342 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1206 14:25:40.660609 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:40.660723 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] }] } I1206 14:25:40.660823 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:40.660912 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:40.660970 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:40.660990 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I1206 14:25:40.661100 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.661182 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:40.661244 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.661284 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I1206 14:25:40.661391 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:40.661441 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:40.661292 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1206 14:25:40.667573 35400 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1206 14:25:40.667796 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.667994 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.668125 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.668301 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.668517 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" }] }] } I1206 14:25:40.668809 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I1206 14:25:40.669048 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.669204 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.669282 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.669397 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:40.669502 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:40.669569 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:40.669614 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:40.669679 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.669844 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.669985 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:40.670113 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I1206 14:25:40.670293 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.670455 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.670733 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.671080 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.671273 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.671430 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.671567 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.671716 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I1206 14:25:40.673302 35400 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1206 14:25:40.673655 35400 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1206 14:25:40.673958 35400 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1206 14:25:40.674043 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.674107 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:40.674198 35400 dns.go:238] Pre-creating DNS records I1206 14:25:40.674259 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:40.681016 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:40.684056 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.684159 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.722715 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.824379 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.825737 35400 executor.go:111] Tasks: 0 done / 94 total; 50 can run I1206 14:25:40.825930 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] }] } I1206 14:25:40.825955 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1206 14:25:40.826042 35400 api.go:53] PageSize not implemented I1206 14:25:40.826156 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:40.826072 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.826071 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.827657 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:40.827766 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:40.827726 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I1206 14:25:40.827834 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:40.827873 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:40.827911 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.827873 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I1206 14:25:40.827957 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W1206 14:25:40.828845 35400 api.go:53] PageSize not implemented W1206 14:25:40.829560 35400 api.go:53] PageSize not implemented I1206 14:25:40.829769 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.830052 35400 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:40.830237 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:40.830317 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:40.831652 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:40.833256 35400 executor.go:111] Tasks: 50 done / 94 total; 19 can run I1206 14:25:40.833394 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I1206 14:25:40.833458 35400 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I1206 14:25:40.833405 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.833591 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I1206 14:25:40.833806 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.833858 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.834033 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:40.834186 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I1206 14:25:40.834485 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.834568 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.834793 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.835012 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:40.838812 35400 executor.go:111] Tasks: 69 done / 94 total; 21 can run I1206 14:25:40.838949 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.839062 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.839179 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.839570 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.839821 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.840073 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.840352 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.842394 35400 executor.go:111] Tasks: 90 done / 94 total; 2 can run I1206 14:25:40.842745 35400 executor.go:111] Tasks: 92 done / 94 total; 2 can run I1206 14:25:40.842938 35400 executor.go:111] Tasks: 94 done / 94 total; 0 can run I1206 14:25:40.846383 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:40.846450 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:40.846487 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:40.846520 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:40.846555 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:40.846594 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:40.846629 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:40.846659 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:40.846695 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:40.846735 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:40.846781 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:40.846822 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:40.846865 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:40.846900 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:40.846933 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:40.846969 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:40.847001 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:40.847035 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:40.847070 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:40.847109 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:40.847142 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:40.847174 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:40.847206 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:40.847242 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:40.847279 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:40.847313 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:40.848716 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:40.848748 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:40.848787 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.848836 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1206 14:25:40.848892 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.848951 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.848999 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.849053 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.849106 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.849146 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I1206 14:25:40.849195 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.849249 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I1206 14:25:40.849289 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W1206 14:25:40.849344 35400 api.go:53] PageSize not implemented I1206 14:25:40.849365 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:40.849420 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:40.849437 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:40.849461 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:40.849476 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:40.849528 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:40.849555 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:40.849604 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:40.849638 35400 iamrole.go:89] ListRoles: { } I1206 14:25:40.849678 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:40.849722 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:40.849755 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:40.850152 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:nodes.externallb.example.com ok I1206 14:25:40.850247 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I1206 14:25:40.850308 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I1206 14:25:40.850227 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:40.850345 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1206 14:25:40.850388 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I1206 14:25:40.850384 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:40.850346 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I1206 14:25:40.850487 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1206 14:25:40.850696 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:40.850717 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I1206 14:25:40.850775 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:40.850837 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I1206 14:25:40.850871 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:40.850924 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:40.851045 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:40.851066 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:40.851106 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1206 14:25:40.851197 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I1206 14:25:40.851248 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I1206 14:25:40.851228 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:40.851263 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I1206 14:25:40.851332 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I1206 14:25:40.851370 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I1206 14:25:40.851421 35400 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I1206 14:25:40.851434 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I1206 14:25:40.851463 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I1206 14:25:40.851517 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:40.851592 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:40.851702 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:40.851980 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.38s) === RUN TestLifecycleSharedSubnet I1206 14:25:40.852275 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:40.852565 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:40.852615 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:40.852655 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:40.852703 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:40.852743 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:40.852770 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:40.852803 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:40.852828 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:40.852900 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I1206 14:25:40.881083 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:40.884207 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.884308 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:40.923759 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.035751 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.037363 35400 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1206 14:25:41.037761 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I1206 14:25:41.037797 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.037849 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:41.037988 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:41.038110 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:41.038148 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:41.037913 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1206 14:25:41.038217 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.038410 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.038554 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.038648 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:41.038707 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I1206 14:25:41.038769 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.038860 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.038958 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.039285 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:41.039396 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.041167 35400 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1206 14:25:41.041430 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1206 14:25:41.041437 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1206 14:25:41.041571 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1206 14:25:41.041635 35400 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1206 14:25:41.041689 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:41.041758 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.041753 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:41.041959 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1206 14:25:41.042200 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.042323 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:41.042194 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.042376 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.042427 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1206 14:25:41.042628 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.042719 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:41.042767 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.042820 35400 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I1206 14:25:41.043778 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1206 14:25:41.046296 35400 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1206 14:25:41.046498 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.047126 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.047394 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I1206 14:25:41.047621 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.047798 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.047930 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.048057 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.048183 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.048340 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.048477 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" }] }] } I1206 14:25:41.048612 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:41.048742 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.048853 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.048992 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.049190 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I1206 14:25:41.049343 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.049450 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.049544 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I1206 14:25:41.051214 35400 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1206 14:25:41.052459 35400 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1206 14:25:41.052747 35400 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1206 14:25:41.052852 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.052918 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.053011 35400 dns.go:238] Pre-creating DNS records I1206 14:25:41.053091 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.062371 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:41.065811 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.065913 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.103876 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.207038 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.208323 35400 executor.go:111] Tasks: 0 done / 80 total; 42 can run I1206 14:25:41.208629 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:41.208708 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.208991 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.210629 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.210796 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.211111 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:41.211241 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:41.211277 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:41.215410 35400 executor.go:111] Tasks: 42 done / 80 total; 16 can run I1206 14:25:41.215728 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1206 14:25:41.215651 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I1206 14:25:41.215863 35400 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1206 14:25:41.215823 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1206 14:25:41.216066 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I1206 14:25:41.216352 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.216441 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.216592 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:41.221336 35400 executor.go:111] Tasks: 58 done / 80 total; 18 can run I1206 14:25:41.221834 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.222167 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.222353 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.222546 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:41.225262 35400 executor.go:111] Tasks: 76 done / 80 total; 2 can run I1206 14:25:41.225588 35400 executor.go:111] Tasks: 78 done / 80 total; 2 can run I1206 14:25:41.225762 35400 executor.go:111] Tasks: 80 done / 80 total; 0 can run I1206 14:25:41.228715 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:41.228786 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:41.228823 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:41.228854 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:41.228888 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:41.228921 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:41.228953 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:41.228985 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:41.229020 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:41.229058 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:41.229098 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:41.229131 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:41.229162 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:41.229191 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:41.229225 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:41.229260 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:41.229297 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:41.229330 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:41.229363 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:41.229399 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:41.229434 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:41.230895 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:41.230927 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:41.230968 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231020 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1206 14:25:41.231077 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231136 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231179 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231222 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231264 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231299 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I1206 14:25:41.231337 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231384 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I1206 14:25:41.231422 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W1206 14:25:41.231472 35400 api.go:53] PageSize not implemented I1206 14:25:41.231494 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:41.231521 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:41.231539 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:41.231563 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:41.231581 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:41.231658 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.231689 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.231746 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:41.231783 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.231831 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:41.231871 35400 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.sharedsubnet.example.com ok I1206 14:25:41.232262 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.232326 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I1206 14:25:41.232308 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I1206 14:25:41.232389 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1206 14:25:41.232443 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I1206 14:25:41.232408 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } I1206 14:25:41.232407 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W1206 14:25:41.232641 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I1206 14:25:41.232666 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:41.232710 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:41.232781 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I1206 14:25:41.232840 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I1206 14:25:41.232915 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.232962 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:41.233072 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.233085 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:41.233159 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I1206 14:25:41.233215 35400 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I1206 14:25:41.233231 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I1206 14:25:41.233265 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I1206 14:25:41.233299 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.233340 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I1206 14:25:41.233353 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I1206 14:25:41.233379 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I1206 14:25:41.233642 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedSubnet I1206 14:25:41.233946 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:41.234207 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:41.234257 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:41.234300 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:41.234354 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:41.234398 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:41.234428 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:41.234466 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:41.234496 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:41.234562 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I1206 14:25:41.263167 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:41.266630 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.266728 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.266795 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.304633 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.419518 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.422452 35400 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1206 14:25:41.424243 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.424443 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I1206 14:25:41.422718 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:41.425397 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I1206 14:25:41.425505 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:41.425714 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:41.425600 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I1206 14:25:41.425785 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.425983 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.426169 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:41.426128 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1206 14:25:41.426328 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.426447 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.426575 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.426895 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:41.427022 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.428068 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.429032 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.429203 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:41.430625 35400 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1206 14:25:41.430831 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1206 14:25:41.430754 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:41.430998 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1206 14:25:41.430910 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1206 14:25:41.431091 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1206 14:25:41.431089 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1206 14:25:41.431255 35400 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1206 14:25:41.431341 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.431373 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.431488 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:41.431546 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.431499 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1206 14:25:41.431624 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1206 14:25:41.431767 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.431837 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:41.431889 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.431896 35400 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I1206 14:25:41.431989 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.432089 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:41.432137 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.432172 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1206 14:25:41.432286 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1206 14:25:41.432414 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1206 14:25:41.432525 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1206 14:25:41.432601 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.432694 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:41.432740 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:41.432781 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:41.432886 35400 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I1206 14:25:41.432999 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.433126 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:41.433181 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:41.433234 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.433320 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:41.433367 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:41.433823 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.436330 35400 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1206 14:25:41.436745 35400 api.go:53] PageSize not implemented W1206 14:25:41.436980 35400 api.go:53] PageSize not implemented I1206 14:25:41.436636 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.437059 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1206 14:25:41.437156 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1206 14:25:41.437291 35400 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I1206 14:25:41.437468 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1206 14:25:41.437526 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1206 14:25:41.437511 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.437603 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1206 14:25:41.437674 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.437718 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1206 14:25:41.437810 35400 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] } I1206 14:25:41.437875 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I1206 14:25:41.437922 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1206 14:25:41.437741 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" }] }] } I1206 14:25:41.437971 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1206 14:25:41.438018 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.438309 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I1206 14:25:41.438490 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.438628 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.438757 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I1206 14:25:41.438901 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.439039 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.439177 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:41.439288 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1206 14:25:41.439492 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.439666 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.439834 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.439942 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.440011 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.440183 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1206 14:25:41.440333 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } I1206 14:25:41.440479 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" }] }] } I1206 14:25:41.440635 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.440771 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.440900 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1206 14:25:41.441028 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:41.441139 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I1206 14:25:41.441315 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I1206 14:25:41.441464 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.441569 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.443911 35400 executor.go:111] Tasks: 100 done / 107 total; 4 can run I1206 14:25:41.444115 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:41.444303 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:41.444473 35400 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1206 14:25:41.444727 35400 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1206 14:25:41.444799 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.444853 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.444909 35400 dns.go:238] Pre-creating DNS records I1206 14:25:41.444954 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.451505 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:41.454844 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.454953 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.455026 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.493450 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.602220 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.604487 35400 executor.go:111] Tasks: 0 done / 107 total; 45 can run I1206 14:25:41.604850 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.605109 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.605264 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.605434 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:41.605464 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:41.605608 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:41.605648 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:41.605717 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.608907 35400 executor.go:111] Tasks: 45 done / 107 total; 24 can run I1206 14:25:41.609110 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1206 14:25:41.609214 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I1206 14:25:41.609260 35400 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1206 14:25:41.609154 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I1206 14:25:41.609562 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.609692 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:41.609881 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I1206 14:25:41.610093 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I1206 14:25:41.610343 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:41.610417 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I1206 14:25:41.610741 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I1206 14:25:41.610941 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I1206 14:25:41.611190 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I1206 14:25:41.611442 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:41.611579 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:41.611824 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.617079 35400 executor.go:111] Tasks: 69 done / 107 total; 31 can run W1206 14:25:41.617502 35400 api.go:53] PageSize not implemented I1206 14:25:41.617521 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.617536 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1206 14:25:41.617709 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } W1206 14:25:41.617788 35400 api.go:53] PageSize not implemented I1206 14:25:41.617755 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.617834 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1206 14:25:41.617891 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.617965 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I1206 14:25:41.618100 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I1206 14:25:41.618155 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I1206 14:25:41.618217 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.618422 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.618684 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.621016 35400 executor.go:111] Tasks: 100 done / 107 total; 4 can run W1206 14:25:41.621136 35400 api.go:53] PageSize not implemented I1206 14:25:41.621102 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:41.621187 35400 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I1206 14:25:41.621174 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I1206 14:25:41.621366 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W1206 14:25:41.621441 35400 api.go:53] PageSize not implemented I1206 14:25:41.621471 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1206 14:25:41.621742 35400 executor.go:111] Tasks: 104 done / 107 total; 3 can run I1206 14:25:41.621976 35400 executor.go:111] Tasks: 107 done / 107 total; 0 can run I1206 14:25:41.625945 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:41.626007 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:41.626044 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:41.626076 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:41.626110 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:41.626143 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:41.626175 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:41.626207 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:41.626243 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:41.626280 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:41.626319 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1206 14:25:41.626362 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:41.626395 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:41.626425 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:41.626454 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:41.626486 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:41.626521 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:41.626554 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:41.626591 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:41.626625 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:41.626661 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:41.626695 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:41.626727 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:41.626761 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:41.626797 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:41.628879 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:41.628908 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:41.628945 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629010 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1206 14:25:41.629089 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629147 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629191 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629233 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629274 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629309 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I1206 14:25:41.629346 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629392 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I1206 14:25:41.629429 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W1206 14:25:41.629474 35400 api.go:53] PageSize not implemented I1206 14:25:41.629491 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I1206 14:25:41.629544 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:41.629571 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:41.629588 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:41.629612 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:41.629628 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I1206 14:25:41.629679 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.629704 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.629750 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:41.629786 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.629829 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:41.629869 35400 routetable.go:53] DescribeRouteTables: { } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok autoscaling-group:nodes.private-shared-subnet.example.com ok I1206 14:25:41.630364 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:41.630430 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1206 14:25:41.630568 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.630517 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } autoscaling-config:lt-3 ok I1206 14:25:41.630475 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I1206 14:25:41.630592 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I1206 14:25:41.630605 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I1206 14:25:41.630438 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.630652 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok security-group:sg-4 ok I1206 14:25:41.630635 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.630685 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:bastion.private-shared-subnet.example.com ok autoscaling-config:lt-2 ok I1206 14:25:41.630724 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } load-balancer:api-private-shared-subnet-n2f8ak ok volume:vol-2 ok I1206 14:25:41.630746 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } iam-instance-profile:masters.private-shared-subnet.example.com ok volume:vol-1 ok autoscaling-config:lt-1 ok I1206 14:25:41.630796 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I1206 14:25:41.630822 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.630894 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.630990 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.630904 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } iam-instance-profile:bastions.private-shared-subnet.example.com ok W1206 14:25:41.631046 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.631085 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:41.631140 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:41.631224 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1206 14:25:41.631574 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.631596 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1206 14:25:41.631728 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:41.631773 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I1206 14:25:41.631859 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I1206 14:25:41.631792 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } I1206 14:25:41.631878 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } W1206 14:25:41.631906 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.631921 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1206 14:25:41.631920 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I1206 14:25:41.631974 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.631955 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.632029 35400 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I1206 14:25:41.632045 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I1206 14:25:41.632076 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I1206 14:25:41.632054 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:41.632169 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:41.632116 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.632184 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I1206 14:25:41.632199 35400 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com security-group:sg-1 ok I1206 14:25:41.632219 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I1206 14:25:41.632251 35400 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I1206 14:25:41.632572 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.40s) === RUN TestLifecyclePrivateSharedIP I1206 14:25:41.632920 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:41.633229 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:41.633286 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:41.633327 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:41.633385 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:41.633432 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:41.633463 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:41.633502 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:41.633534 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:41.633601 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I1206 14:25:41.662170 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:41.665610 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.665704 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.665767 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.704837 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.817483 35400 iamrole.go:89] ListRoles: { } I1206 14:25:41.819828 35400 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1206 14:25:41.820033 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:41.820168 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:41.820213 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:41.820268 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.820415 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.820650 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.820659 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:41.820799 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:41.820737 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I1206 14:25:41.820855 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:41.820888 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1206 14:25:41.820979 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:41.821026 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.821078 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:41.821193 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I1206 14:25:41.821215 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.821300 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1206 14:25:41.821385 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I1206 14:25:41.821458 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I1206 14:25:41.823293 35400 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1206 14:25:41.823431 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1206 14:25:41.823554 35400 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1206 14:25:41.823499 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.823621 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1206 14:25:41.823625 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:41.823694 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1206 14:25:41.823745 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1206 14:25:41.823759 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I1206 14:25:41.823893 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1206 14:25:41.823860 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.824038 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1206 14:25:41.824103 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I1206 14:25:41.824127 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1206 14:25:41.824209 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1206 14:25:41.824365 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.824471 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:41.824519 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:41.824561 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:41.824613 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:41.824541 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1206 14:25:41.824702 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1206 14:25:41.824977 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1206 14:25:41.825238 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.825347 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:41.825444 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.825526 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.825622 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1206 14:25:41.825755 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:41.825848 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.825938 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:41.825996 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:41.826039 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:41.825817 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I1206 14:25:41.826086 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:41.826135 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:41.826184 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:41.826293 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:41.825988 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1206 14:25:41.826356 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I1206 14:25:41.826492 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:41.826552 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:41.826598 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:41.826653 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:41.826691 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:41.826739 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I1206 14:25:41.826873 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I1206 14:25:41.826963 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:41.827009 35400 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I1206 14:25:41.829409 35400 executor.go:111] Tasks: 71 done / 116 total; 36 can run W1206 14:25:41.829719 35400 api.go:53] PageSize not implemented I1206 14:25:41.829622 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } W1206 14:25:41.829777 35400 api.go:53] PageSize not implemented I1206 14:25:41.829864 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1206 14:25:41.829822 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.829977 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1206 14:25:41.829969 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.830084 35400 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1206 14:25:41.830166 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1206 14:25:41.830202 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1206 14:25:41.830179 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:41.830263 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1206 14:25:41.830336 35400 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I1206 14:25:41.830395 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1206 14:25:41.830427 35400 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1206 14:25:41.830474 35400 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1206 14:25:41.830419 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" }] }] } I1206 14:25:41.830706 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I1206 14:25:41.830830 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.830976 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:41.831813 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1206 14:25:41.832067 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:41.832228 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:41.832373 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" }] }] } I1206 14:25:41.832739 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:41.832916 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.833209 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.833402 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1206 14:25:41.833656 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1206 14:25:41.833930 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1206 14:25:41.834202 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:41.834304 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I1206 14:25:41.834543 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" }] }] } I1206 14:25:41.835649 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.835756 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I1206 14:25:41.835881 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I1206 14:25:41.835978 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.836154 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.836285 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I1206 14:25:41.837283 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1206 14:25:41.837338 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I1206 14:25:41.837375 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I1206 14:25:41.837522 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1206 14:25:41.837671 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.837796 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.837946 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I1206 14:25:41.838079 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:41.838123 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I1206 14:25:41.838257 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.838370 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.838413 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I1206 14:25:41.838450 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:41.838489 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.838531 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:41.838618 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.838787 35400 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I1206 14:25:41.840921 35400 executor.go:111] Tasks: 107 done / 116 total; 5 can run I1206 14:25:41.841046 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:41.841182 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:41.841210 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:41.841286 35400 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:41.841457 35400 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I1206 14:25:41.841557 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:41.841625 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:41.841674 35400 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I1206 14:25:41.841746 35400 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1206 14:25:41.841816 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:41.841944 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I1206 14:25:41.842014 35400 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1206 14:25:41.842064 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:41.842101 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.842149 35400 dns.go:238] Pre-creating DNS records I1206 14:25:41.842175 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:41.849432 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I1206 14:25:41.852864 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.852958 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.853025 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:41.890592 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.001546 35400 iamrole.go:89] ListRoles: { } I1206 14:25:42.006107 35400 executor.go:111] Tasks: 0 done / 116 total; 45 can run I1206 14:25:42.006519 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:42.006782 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] }] } I1206 14:25:42.007025 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I1206 14:25:42.007137 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I1206 14:25:42.007191 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I1206 14:25:42.007282 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.007458 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.007554 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.009653 35400 executor.go:111] Tasks: 45 done / 116 total; 26 can run I1206 14:25:42.010027 35400 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1206 14:25:42.009942 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I1206 14:25:42.010101 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1206 14:25:42.010181 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.010321 35400 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1206 14:25:42.010366 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:42.010432 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:42.010500 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I1206 14:25:42.010640 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I1206 14:25:42.010800 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:42.010866 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.011134 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I1206 14:25:42.011206 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.011387 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I1206 14:25:42.011590 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I1206 14:25:42.011826 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I1206 14:25:42.012033 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I1206 14:25:42.012169 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:42.012298 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:42.012400 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:42.014869 35400 executor.go:111] Tasks: 71 done / 116 total; 36 can run W1206 14:25:42.015074 35400 api.go:53] PageSize not implemented I1206 14:25:42.015106 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.015117 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } W1206 14:25:42.015275 35400 api.go:53] PageSize not implemented I1206 14:25:42.015236 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1206 14:25:42.015332 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:42.015324 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1206 14:25:42.015455 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1206 14:25:42.015426 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.015500 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1206 14:25:42.015540 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.015562 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I1206 14:25:42.015649 35400 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I1206 14:25:42.015752 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.015939 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.016112 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.016304 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.016421 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:42.016575 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.016771 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I1206 14:25:42.019074 35400 executor.go:111] Tasks: 107 done / 116 total; 5 can run W1206 14:25:42.019204 35400 api.go:53] PageSize not implemented I1206 14:25:42.019237 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1206 14:25:42.019268 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I1206 14:25:42.019337 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:42.019457 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W1206 14:25:42.019480 35400 api.go:53] PageSize not implemented I1206 14:25:42.019406 35400 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I1206 14:25:42.019522 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I1206 14:25:42.019757 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I1206 14:25:42.019861 35400 executor.go:111] Tasks: 112 done / 116 total; 4 can run I1206 14:25:42.019938 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I1206 14:25:42.020113 35400 executor.go:111] Tasks: 116 done / 116 total; 0 can run I1206 14:25:42.023598 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:42.023659 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:42.023700 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:42.023732 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:42.023767 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:42.023807 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:42.023840 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:42.023871 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:42.023907 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:42.023944 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:42.023981 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I1206 14:25:42.024023 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I1206 14:25:42.024058 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:42.024089 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:42.024121 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:42.024156 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:42.024191 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I1206 14:25:42.024226 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:42.024263 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:42.024297 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I1206 14:25:42.024332 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I1206 14:25:42.024366 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I1206 14:25:42.024401 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:42.024438 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:42.024471 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I1206 14:25:42.024511 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:42.024545 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:42.024577 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:42.024612 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:42.024648 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:42.026208 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:42.026239 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:42.026282 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026351 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1206 14:25:42.026430 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026496 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026539 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026581 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026624 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026670 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I1206 14:25:42.026723 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026778 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:42.026806 35400 natgateway.go:117] DescribeNatGateways: { } I1206 14:25:42.026848 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I1206 14:25:42.026885 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W1206 14:25:42.026931 35400 api.go:53] PageSize not implemented I1206 14:25:42.026947 35400 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I1206 14:25:42.026996 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:42.027022 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:42.027041 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:42.027064 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:42.027082 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:42.027136 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.027164 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:42.027206 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:42.027247 35400 iamrole.go:89] ListRoles: { } I1206 14:25:42.027296 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:42.027346 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:42.027378 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I1206 14:25:42.027414 35400 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I1206 14:25:42.027447 35400 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I1206 14:25:42.028017 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } autoscaling-group:nodes.private-shared-ip.example.com ok autoscaling-group:bastion.private-shared-ip.example.com ok I1206 14:25:42.028083 35400 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I1206 14:25:42.028051 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I1206 14:25:42.028149 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } load-balancer:api-private-shared-ip-exa-ohatqj ok load-balancer:bastion-private-shared-ip-eepmph ok I1206 14:25:42.028202 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I1206 14:25:42.028161 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:42.028240 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1206 14:25:42.028295 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I1206 14:25:42.028155 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-ip.example.com ok I1206 14:25:42.028366 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I1206 14:25:42.028460 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1206 14:25:42.028390 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W1206 14:25:42.028554 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.028589 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I1206 14:25:42.028516 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I1206 14:25:42.028687 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:42.028769 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W1206 14:25:42.028914 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.028946 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I1206 14:25:42.029035 35400 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I1206 14:25:42.029122 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I1206 14:25:42.029218 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I1206 14:25:42.029294 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W1206 14:25:42.029512 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.029534 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I1206 14:25:42.029652 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I1206 14:25:42.029667 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:42.029721 35400 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I1206 14:25:42.029733 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:42.029752 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I1206 14:25:42.029843 35400 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I1206 14:25:42.029865 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I1206 14:25:42.029801 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W1206 14:25:42.029915 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.029908 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } I1206 14:25:42.029939 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } iam-role:masters.private-shared-ip.example.com ok security-group:sg-2 ok I1206 14:25:42.029966 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I1206 14:25:42.030004 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I1206 14:25:42.030078 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I1206 14:25:42.030020 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W1206 14:25:42.030102 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.030096 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I1206 14:25:42.030117 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I1206 14:25:42.030130 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I1206 14:25:42.030160 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I1206 14:25:42.030193 35400 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I1206 14:25:42.030149 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I1206 14:25:42.030315 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I1206 14:25:42.030403 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:42.030675 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I1206 14:25:42.030947 35400 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:25:42.031227 35400 internetgateways.go:66] CreateInternetGateway: { } I1206 14:25:42.031275 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:25:42.031317 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:25:42.031362 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:25:42.031401 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:25:42.031432 35400 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:25:42.031471 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:25:42.031498 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:25:42.031561 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I1206 14:25:42.056466 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1206 14:25:42.059564 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.059654 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.098889 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.226169 35400 iamrole.go:89] ListRoles: { } I1206 14:25:42.229640 35400 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1206 14:25:42.229951 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.230029 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.230125 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.230158 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.230290 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1206 14:25:42.230326 35400 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I1206 14:25:42.230313 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.230475 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.230618 35400 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I1206 14:25:42.230663 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:42.230743 35400 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I1206 14:25:42.230842 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I1206 14:25:42.230942 35400 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I1206 14:25:42.231022 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:42.231081 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.231132 35400 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.231266 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:42.231312 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:42.231592 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:42.231703 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.231884 35400 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.232161 35400 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I1206 14:25:42.232375 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:42.233300 35400 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1206 14:25:42.233430 35400 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1206 14:25:42.233473 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.233614 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1206 14:25:42.233647 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.233807 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I1206 14:25:42.233921 35400 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.234041 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1206 14:25:42.234020 35400 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I1206 14:25:42.234033 35400 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.234216 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:42.234301 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.234526 35400 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:25:42.234615 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.234658 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1206 14:25:42.234906 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.234907 35400 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" } ] }], VpcId: "vpc-1" } I1206 14:25:42.235106 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:42.235175 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1206 14:25:42.235305 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1206 14:25:42.235422 35400 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I1206 14:25:42.235489 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:42.235543 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:42.235600 35400 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I1206 14:25:42.235691 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:42.235875 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:42.235983 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:42.236055 35400 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I1206 14:25:42.236218 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:42.236297 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:42.237062 35400 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.240292 35400 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1206 14:25:42.240549 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.240813 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.241015 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.241243 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.241460 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1206 14:25:42.241716 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.241892 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.241966 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.242037 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.242101 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.242239 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.242407 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.242552 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1206 14:25:42.242685 35400 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.242820 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.242930 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.243072 35400 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:42.243170 35400 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I1206 14:25:42.243221 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I1206 14:25:42.243266 35400 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I1206 14:25:42.243310 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1206 14:25:42.243495 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I1206 14:25:42.243643 35400 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I1206 14:25:42.243796 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.243906 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.245176 35400 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1206 14:25:42.245439 35400 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1206 14:25:42.245676 35400 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1206 14:25:42.245723 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.245757 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:42.245808 35400 dns.go:238] Pre-creating DNS records I1206 14:25:42.245838 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:42.251467 35400 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I1206 14:25:42.254521 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.254613 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.292750 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.421555 35400 iamrole.go:89] ListRoles: { } I1206 14:25:42.423870 35400 executor.go:111] Tasks: 0 done / 101 total; 45 can run I1206 14:25:42.424060 35400 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I1206 14:25:42.424395 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.424510 35400 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I1206 14:25:42.424664 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.424799 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:25:42.424853 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.425185 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.425322 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.425446 35400 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:25:42.425499 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.428737 35400 executor.go:111] Tasks: 45 done / 101 total; 24 can run I1206 14:25:42.429034 35400 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1206 14:25:42.429070 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.429267 35400 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1206 14:25:42.429337 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I1206 14:25:42.429791 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I1206 14:25:42.430820 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:42.431595 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.431855 35400 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:25:42.431922 35400 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:25:42.432065 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.432214 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.432344 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:42.436718 35400 executor.go:111] Tasks: 69 done / 101 total; 26 can run I1206 14:25:42.436980 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.437246 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.437453 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.437550 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I1206 14:25:42.437738 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.437922 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.438153 35400 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I1206 14:25:42.440444 35400 executor.go:111] Tasks: 95 done / 101 total; 2 can run I1206 14:25:42.440685 35400 executor.go:111] Tasks: 97 done / 101 total; 4 can run I1206 14:25:42.440907 35400 executor.go:111] Tasks: 101 done / 101 total; 0 can run I1206 14:25:42.443997 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I1206 14:25:42.444067 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I1206 14:25:42.444101 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I1206 14:25:42.444132 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I1206 14:25:42.444163 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I1206 14:25:42.444202 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I1206 14:25:42.444236 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I1206 14:25:42.444265 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I1206 14:25:42.444296 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I1206 14:25:42.444330 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I1206 14:25:42.444363 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I1206 14:25:42.444400 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I1206 14:25:42.444442 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I1206 14:25:42.444473 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I1206 14:25:42.444504 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I1206 14:25:42.444542 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I1206 14:25:42.444574 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:25:42.444607 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I1206 14:25:42.444640 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:25:42.444676 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I1206 14:25:42.444708 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I1206 14:25:42.444738 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I1206 14:25:42.444769 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I1206 14:25:42.444804 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I1206 14:25:42.444838 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:25:42.444875 35400 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W1206 14:25:42.446382 35400 instances.go:27] MockEc2::DescribeInstances is stub-implemented I1206 14:25:42.446412 35400 keypairs.go:95] DescribeKeyPairs: { } I1206 14:25:42.446450 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446506 35400 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446563 35400 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446627 35400 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446679 35400 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446736 35400 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446777 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446816 35400 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446865 35400 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446918 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I1206 14:25:42.446958 35400 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W1206 14:25:42.447013 35400 api.go:53] PageSize not implemented I1206 14:25:42.447036 35400 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W1206 14:25:42.447063 35400 loadbalancers.go:34] PageSize not implemented I1206 14:25:42.447084 35400 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W1206 14:25:42.447108 35400 targetgroups.go:35] PageSize not implemented I1206 14:25:42.447127 35400 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I1206 14:25:42.447180 35400 zones.go:93] ListHostedZonesPages { } I1206 14:25:42.447210 35400 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I1206 14:25:42.447260 35400 iaminstanceprofile.go:193] ListInstanceProfiles: { } I1206 14:25:42.447299 35400 iamrole.go:89] ListRoles: { } I1206 14:25:42.447355 35400 internetgateways.go:101] DescribeInternetGateways: { } I1206 14:25:42.447398 35400 routetable.go:53] DescribeRouteTables: { } I1206 14:25:42.447433 35400 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I1206 14:25:42.447879 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I1206 14:25:42.447983 35400 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I1206 14:25:42.448069 35400 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I1206 14:25:42.448088 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.448173 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } autoscaling-config:lt-1 ok I1206 14:25:42.448021 35400 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I1206 14:25:42.448207 35400 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I1206 14:25:42.448138 35400 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } I1206 14:25:42.448257 35400 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } internet-gateway:igw-2 ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok I1206 14:25:42.448305 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I1206 14:25:42.448380 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W1206 14:25:42.448597 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.448622 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I1206 14:25:42.448659 35400 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I1206 14:25:42.448716 35400 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W1206 14:25:42.448832 35400 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I1206 14:25:42.448847 35400 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I1206 14:25:42.448887 35400 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I1206 14:25:42.448982 35400 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I1206 14:25:42.449003 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.449101 35400 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I1206 14:25:42.449120 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I1206 14:25:42.449179 35400 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I1206 14:25:42.449230 35400 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1206 14:25:42.449313 35400 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I1206 14:25:42.449329 35400 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I1206 14:25:42.449363 35400 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I1206 14:25:42.449437 35400 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I1206 14:25:42.449521 35400 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I1206 14:25:42.449598 35400 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I1206 14:25:42.449879 35400 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 48.915s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.014s ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I1206 14:24:34.499512 26695 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.065s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.056s === RUN TestPodController I1206 14:24:39.140497 31009 pod.go:59] starting pod controller I1206 14:24:39.140802 31009 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.096s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.036s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011fdd0} route53_test.go:93: Got zone list: [0xc00011fe00] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126080} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011fed0} route53_test.go:93: Got zone list: [0xc00011ff00] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126090} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00011ffa0} route53_test.go:170: Successfully added managed DNS zone: &{0xc000341800 0xc0001260a8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036a100} route53_test.go:93: Got zone list: [0xc00036a130] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc0001260c0} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036a230} route53_test.go:93: Got zone list: [0xc00036a260] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc0001260e0} route53_test.go:187: Successfully added resource record set: {0xc00018a600 0xc000126100} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036a4e0} route53_test.go:93: Got zone list: [0xc00036a510] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126188} route53_test.go:199: Successfully added resource record set: {0xc00018a780 0xc0001261a8} route53_test.go:118: Got 1 recordsets: [0xc00036a6b0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036a7e0} route53_test.go:93: Got zone list: [0xc00036a810] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126240} route53_test.go:221: Successfully added resource record set: {0xc00018a900 0xc000126260} route53_test.go:228: Correctly failed to add duplicate resource record {0xc00018a900 0xc000126260}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036ab70} route53_test.go:93: Got zone list: [0xc00036aba0] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126318} route53_test.go:246: Successfully removed resource set {0xc00018ab00 0xc000126338} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036ae20} route53_test.go:93: Got zone list: [0xc00036ae50] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc0001263c0} route53_test.go:264: Successfully removed resource set {0xc00018ac80 0xc0001263e0} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036b100} route53_test.go:93: Got zone list: [0xc00036b130] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126470} commontests.go:192: Successfully added resource record set: {0xc00018ae00 0xc000126490} commontests.go:45: Correctly replaced resource record {0xc00018ae00 0xc000126490} -> {0xc00018af00 0xc0001264e0} commontests.go:142: Got 1 recordsets: [0xc00036b440] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036b610} route53_test.go:93: Got zone list: [0xc00036b640] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc000126590} commontests.go:192: Successfully added resource record set: {0xc00018b180 0xc0001265b0} commontests.go:70: Correctly replaced resource record {0xc00018b180 0xc0001265b0} -> {0xc00018b280 0xc000126600} commontests.go:142: Got 1 recordsets: [0xc00036b950] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036bb70} route53_test.go:93: Got zone list: [0xc00036bba0] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc0001266c0} commontests.go:192: Successfully added resource record set: {0xc00018b500 0xc0001266e0} commontests.go:142: Got 2 recordsets: [0xc00036be10 0xc00036be30] commontests.go:142: Got 2 recordsets: [0xc00036bea0 0xc00036bec0] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00037c060} route53_test.go:93: Got zone list: [0xc00037c090] route53_test.go:98: Got at least 1 zone in list:&{0xc000341620 0xc0001267e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.046s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.029s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005dd860} clouddns_test.go:71: Got zone list: [0xc00039bc98] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc00039bc68} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005dd960} clouddns_test.go:71: Got zone list: [0xc00039bd40] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc00039bd10} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005dda60} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc0005d76c0 0xc00039bde8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005ddb20} clouddns_test.go:71: Got zone list: [0xc00039be90] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc00039be60} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005ddc40} clouddns_test.go:71: Got zone list: [0xc00039bfb0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc00039bf80} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005ddd80} clouddns_test.go:71: Got zone list: [0xc0004f6228] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f61c8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0005ddee0} clouddns_test.go:71: Got zone list: [0xc0004f6678] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f6618} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc000186040} clouddns_test.go:71: Got zone list: [0xc0004f6960] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f6900} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc000186180} clouddns_test.go:71: Got zone list: [0xc0004f6c00] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f6ba0} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0001862e0} clouddns_test.go:71: Got zone list: [0xc0004f6f78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f6f18} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc000186440} clouddns_test.go:71: Got zone list: [0xc0004f74e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f7488} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0001865c0} clouddns_test.go:71: Got zone list: [0xc0004f7b30] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0004f7ad0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00001c048 0xc0001867a0} clouddns_test.go:71: Got zone list: [0xc0001d4150] clouddns_test.go:76: Got at least 1 zone in list:&{0xc0005d7580 0xc0001d4120} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.100s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.142s ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.111s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I1206 14:24:54.458046 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.458411 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.458469 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.458529 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.458621 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.458686 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.458718 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.458772 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.458811 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.472474 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I1206 14:24:54.479203 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.479467 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.479514 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.479556 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.479606 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.479648 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.479678 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.479718 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.479746 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.481558 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I1206 14:24:54.487275 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.487514 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.487558 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.487597 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.487643 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.487679 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.487710 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.487747 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.487773 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.490191 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.492883 35468 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I1206 14:24:54.492910 35468 assetstore.go:111] bin/containerd I1206 14:24:54.492917 35468 assetstore.go:111] bin/containerd-shim I1206 14:24:54.492923 35468 assetstore.go:111] bin/containerd-shim-runc-v1 I1206 14:24:54.492930 35468 assetstore.go:111] bin/containerd-shim-runc-v2 I1206 14:24:54.492935 35468 assetstore.go:111] bin/containerd-stress I1206 14:24:54.492941 35468 assetstore.go:111] bin/ctr I1206 14:24:54.492973 35468 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I1206 14:24:54.492984 35468 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I1206 14:24:54.495672 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.495817 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.495848 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.495876 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.495904 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.495929 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.495944 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.495968 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.495983 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.496939 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.498084 35468 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.00s) === RUN TestContainerdBuilder_SkipInstall I1206 14:24:54.499292 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.499484 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.499521 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.499559 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.499605 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.499666 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.499680 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.499703 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.499717 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.500655 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.501890 35468 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I1206 14:24:54.502436 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.502565 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.502593 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.502616 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.502640 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.502678 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.502696 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.502716 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.502729 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.503599 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.504816 35468 assetstore.go:108] Matching assets for "^docker/": I1206 14:24:54.504829 35468 assetstore.go:111] docker/containerd I1206 14:24:54.504837 35468 assetstore.go:111] docker/containerd-shim I1206 14:24:54.504843 35468 assetstore.go:111] docker/ctr I1206 14:24:54.504849 35468 assetstore.go:111] docker/docker I1206 14:24:54.504855 35468 assetstore.go:111] docker/docker-init I1206 14:24:54.504860 35468 assetstore.go:111] docker/docker-proxy I1206 14:24:54.504866 35468 assetstore.go:111] docker/dockerd I1206 14:24:54.504873 35468 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.00s) === RUN TestDockerBuilder_LogFlags I1206 14:24:54.507210 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.507330 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.507357 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.507379 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.507406 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.507452 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.507466 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.507486 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.507500 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.508417 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.509660 35468 assetstore.go:108] Matching assets for "^docker/": I1206 14:24:54.509674 35468 assetstore.go:111] docker/containerd I1206 14:24:54.509681 35468 assetstore.go:111] docker/containerd-shim I1206 14:24:54.509687 35468 assetstore.go:111] docker/ctr I1206 14:24:54.509693 35468 assetstore.go:111] docker/docker I1206 14:24:54.509699 35468 assetstore.go:111] docker/docker-init I1206 14:24:54.509705 35468 assetstore.go:111] docker/docker-proxy I1206 14:24:54.509712 35468 assetstore.go:111] docker/dockerd I1206 14:24:54.509718 35468 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.00s) === RUN TestDockerBuilder_SkipInstall I1206 14:24:54.512006 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.512148 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.512175 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.512199 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.512223 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.512268 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.512284 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.512302 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.512314 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.513209 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.514422 35468 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I1206 14:24:54.514575 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.514685 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.514706 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.514729 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.514752 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.514787 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.514801 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.514819 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.514831 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.515689 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:54.516820 35468 assetstore.go:108] Matching assets for "^docker/": I1206 14:24:54.516833 35468 assetstore.go:111] docker/docker I1206 14:24:54.516840 35468 assetstore.go:111] docker/docker-containerd I1206 14:24:54.516846 35468 assetstore.go:111] docker/docker-containerd-ctr I1206 14:24:54.516852 35468 assetstore.go:111] docker/docker-containerd-shim I1206 14:24:54.516858 35468 assetstore.go:111] docker/docker-init I1206 14:24:54.516864 35468 assetstore.go:111] docker/docker-proxy I1206 14:24:54.516870 35468 assetstore.go:111] docker/docker-runc I1206 14:24:54.516875 35468 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I1206 14:24:54.520658 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.520818 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.520849 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.520873 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.520898 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.520921 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.520935 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.520954 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.520968 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.525464 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I1206 14:24:54.528447 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.528638 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.528677 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.528718 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.528758 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.528794 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.528820 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.528890 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.528914 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.531421 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I1206 14:24:54.534363 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.534545 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.534602 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.534638 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.534678 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.534713 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.534737 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.534771 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.534793 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.537116 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.00s) === RUN TestKubeAPIServerBuilder I1206 14:24:54.545300 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.545531 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.545579 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.545616 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.545657 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.545695 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.545721 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.545753 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.545777 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.548532 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I1206 14:24:54.559655 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.559864 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.559921 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.559970 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.560009 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.560060 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.560086 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.560119 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.560142 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.562614 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I1206 14:24:54.572831 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.573024 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.573066 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.573102 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.573142 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.573176 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.573202 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.573232 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.573254 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.575419 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I1206 14:24:54.583561 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.583756 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.583801 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.583837 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.583880 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.583915 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.583938 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.583969 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.583993 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.586390 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I1206 14:24:54.596521 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:54.596711 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:54.596758 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:54.596794 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:54.596835 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:54.596873 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:54.596896 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:54.596926 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:54.596949 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:54.599317 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (0.55s) === RUN TestKubeAPIServerBuilderARM64 I1206 14:24:55.147650 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.147832 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.147870 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.147901 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.147933 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.147959 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.147976 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.147999 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.148015 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.149805 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I1206 14:24:55.226269 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.226440 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.226475 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.226506 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.226539 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.226564 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.226580 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.226603 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.226619 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.228199 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I1206 14:24:55.234039 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.234173 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.234200 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.234227 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.234253 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.234275 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.234292 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.234311 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.234325 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.235945 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.08s) === RUN TestKubeControllerManagerBuilderARM64 I1206 14:24:55.313876 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.314739 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.314797 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.314843 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.314883 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.314918 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.314952 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.314986 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.315008 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.317512 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I1206 14:24:55.391798 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.391944 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.391982 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.392011 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.392042 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.392065 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.392079 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.392100 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.392116 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.393578 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.00s) === RUN TestKubeProxyBuilderAMD64 I1206 14:24:55.396723 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.396866 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.396897 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.396921 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.396946 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.396971 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.396988 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.397008 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.397037 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.398691 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I1206 14:24:55.468293 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.468456 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.468487 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.468513 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.468540 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.468565 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.468579 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.468603 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.468616 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.470131 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I1206 14:24:55.537732 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.537905 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.537943 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.537975 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.538009 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.538034 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.538052 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.538075 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.538093 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.540300 35468 zones.go:93] ListHostedZonesPages { } W1206 14:24:55.544264 35468 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I1206 14:24:55.546399 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.546569 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.546602 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.546627 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.546656 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.546679 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.546694 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.546714 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.546728 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.548308 35468 zones.go:93] ListHostedZonesPages { } W1206 14:24:55.617934 35468 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I1206 14:24:55.620223 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.620399 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.620442 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.620484 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.620517 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.620551 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.620569 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.620591 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.620613 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.622665 35468 zones.go:93] ListHostedZonesPages { } W1206 14:24:55.690554 35468 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I1206 14:24:55.692611 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.692776 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.692814 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.692841 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.692873 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.692901 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.692918 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.692947 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.692971 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.694639 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.696673 35468 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I1206 14:24:55.698521 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.698635 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.698672 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.698694 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.698717 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.698739 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.698755 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.698775 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.698792 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.700031 35468 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I1206 14:24:55.703255 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.703369 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.703406 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.703429 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.703453 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.703477 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.703499 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.703519 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.703532 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.707162 35468 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I1206 14:24:55.711987 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.712204 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.712242 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.712281 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.712322 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.712359 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.712384 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.712416 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.712439 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.714856 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.717084 35468 assetstore.go:108] Matching assets for "protokube$": I1206 14:24:55.717108 35468 assetstore.go:111] /opt/kops/bin/protokube I1206 14:24:55.717122 35468 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I1206 14:24:55.717159 35468 assetstore.go:108] Matching assets for "channels$": I1206 14:24:55.717173 35468 assetstore.go:111] /opt/kops/bin/channels I1206 14:24:55.717183 35468 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I1206 14:24:55.719258 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.719485 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.719525 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.719562 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.719602 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.719637 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.719661 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.719692 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.719713 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.722115 35468 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I1206 14:24:55.725221 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.725438 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.725475 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.725510 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.725551 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.725582 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.725607 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.725638 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.725660 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.727950 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.730181 35468 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I1206 14:24:55.730899 35468 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:55.731133 35468 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.731176 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:55.731214 35468 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:55.731260 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:55.731294 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:55.731324 35468 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:55.731384 35468 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:55.731412 35468 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:55.733982 35468 zones.go:93] ListHostedZonesPages { } I1206 14:24:55.736420 35468 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 1.373s ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.082s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.071s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.056s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.156s ? k8s.io/kops/pkg/apis/kops/registry [no test files] === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E1206 14:24:37.444135 29356 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E1206 14:24:37.444272 29356 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.026s ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I1206 14:24:46.602280 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602370 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602407 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602443 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602473 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I1206 14:24:46.602680 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602719 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602745 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602781 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602817 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602845 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602880 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602906 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602936 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602965 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.602990 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603019 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603048 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603073 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603115 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603156 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603184 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603210 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I1206 14:24:46.603235 33423 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I1206 14:24:46.603408 33423 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I1206 14:24:46.603463 33423 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.111s ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] === RUN TestApplySet I1206 14:24:40.200569 31464 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1206 14:24:40.201244 31464 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1206 14:24:40.201921 31464 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1206 14:24:40.202370 31464 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1206 14:24:40.203360 31464 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I1206 14:24:40.203638 31464 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I1206 14:24:40.204015 31464 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1206 14:24:40.204386 31464 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I1206 14:24:40.209007 31464 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I1206 14:24:40.209068 31464 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I1206 14:24:40.209315 31464 health.go:33] status conditions not found for /v1, Kind=Namespace I1206 14:24:40.209499 31464 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I1206 14:24:40.209540 31464 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I1206 14:24:40.209746 31464 health.go:33] status conditions not found for /v1, Kind=ConfigMap I1206 14:24:40.209930 31464 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I1206 14:24:40.209976 31464 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I1206 14:24:40.210543 31464 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I1206 14:24:40.210582 31464 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.122s ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W1206 14:24:38.803246 29812 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (1.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W1206 14:24:38.804509 29812 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W1206 14:24:38.805693 29812 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W1206 14:24:38.806818 29812 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W1206 14:24:38.808176 29812 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W1206 14:24:38.809499 29812 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W1206 14:24:38.810707 29812 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E1206 14:24:38.811629 29812 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E1206 14:24:38.811645 29812 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E1206 14:24:38.811656 29812 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E1206 14:24:38.811669 29812 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 1.096s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.006s ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.116s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.097s ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.01s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.01s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.01s) PASS ok k8s.io/kops/pkg/commands 0.226s ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.053s ? k8s.io/kops/pkg/configserver [no test files] === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.086s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.045s ? k8s.io/kops/pkg/dump [no test files] === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.03s) PASS ok k8s.io/kops/pkg/edit 0.080s === RUN TestFlagToFalse I1206 14:24:39.633599 31335 featureflag_test.go:32] Created flag Unittest1 I1206 14:24:39.641723 31335 featureflag.go:160] FeatureFlag "UnitTest1"=false I1206 14:24:39.641743 31335 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.01s) === RUN TestSetenv I1206 14:24:39.641877 31335 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.050s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.058s === RUN TestRenderInstanceGroupZones W1206 14:24:42.025559 31963 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.058s === RUN TestWarmPoolOnlyRoll I1206 14:25:05.196080 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.196177 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.196200 35813 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I1206 14:25:05.196655 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.196681 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.197893 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.197935 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.203146 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.203197 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.203214 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.203224 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.205203 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.205922 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.205952 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.207158 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.207191 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.212458 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.212500 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.213151 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.213179 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.214379 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.214414 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.219663 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.219693 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.219727 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.219740 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.220501 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.221085 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.221113 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.222286 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.222334 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.227587 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.227626 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.228266 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.228308 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.229508 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.229556 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.234817 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.234856 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.235568 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.235597 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.236778 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.236830 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.242151 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.242175 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.242190 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.242201 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.242937 35813 instancegroups.go:407] Draining the node: "node-2a.local". I1206 14:25:05.243572 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.243600 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.244795 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.244841 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.250089 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.250128 35813 instancegroups.go:407] Draining the node: "node-2b.local". I1206 14:25:05.250791 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.250831 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.252570 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.252659 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.257971 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.258038 35813 instancegroups.go:407] Draining the node: "node-2c.local". I1206 14:25:05.259096 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.259141 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.260365 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.260399 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.265647 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.265701 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I1206 14:25:05.266241 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.266277 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.267469 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.267534 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.267572 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.267589 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.267615 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.268782 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.268833 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.268848 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.268876 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.270071 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.270108 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.270155 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.270176 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.270205 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.271400 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.271450 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.271465 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.271493 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.272672 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.272723 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.272738 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.272766 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.273963 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.273990 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.274037 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.274052 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.274079 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.275257 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.275308 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.275324 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.275351 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.276549 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W1206 14:25:05.276601 35813 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I1206 14:25:05.276616 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.276642 35813 instancegroups.go:443] waiting for 1ms after terminating instance W1206 14:25:05.277818 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1206 14:25:05.277850 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I1206 14:25:05.278151 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.278185 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.279404 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.279445 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.279465 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.279512 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.279536 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.279550 35813 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.279564 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.280293 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.281123 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.281160 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.282335 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.282371 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.282390 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.282426 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.283182 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.283229 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.284396 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.284431 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.284449 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.284468 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.284486 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.284498 35813 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.284510 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.285416 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.286122 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.286156 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.287336 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.287371 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.287396 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.287431 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.288209 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.288241 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.289450 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.289492 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.289514 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.289553 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.290449 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.290487 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.291694 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.291743 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.291766 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.291781 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.291800 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.291814 35813 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.291828 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.292819 35813 instancegroups.go:407] Draining the node: "node-2a.local". I1206 14:25:05.293671 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.293710 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.294895 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.294928 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.294958 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.294994 35813 instancegroups.go:407] Draining the node: "node-2b.local". I1206 14:25:05.295856 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.295894 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.297133 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.297174 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.297196 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.297236 35813 instancegroups.go:407] Draining the node: "node-2c.local". I1206 14:25:05.298098 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.298137 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.299355 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.299397 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W1206 14:25:05.299418 35813 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I1206 14:25:05.299436 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I1206 14:25:05.299742 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I1206 14:25:05.300024 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.300059 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.301253 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.301287 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.306517 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.306579 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.306601 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.306614 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.307374 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.308150 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.308188 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.309437 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.309473 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.314737 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.314787 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.315654 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.315694 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.316901 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.316936 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.322225 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.322270 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.322290 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.322304 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.323278 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.324073 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.324111 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.325302 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.325337 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.330594 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.330651 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.331471 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.331510 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.332787 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.332823 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.338143 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.338194 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.339062 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.339102 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.340322 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.340363 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.345631 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.345671 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.345698 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.345712 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.346698 35813 instancegroups.go:407] Draining the node: "node-2a.local". I1206 14:25:05.347446 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.347485 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.348682 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.348718 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.353965 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.354016 35813 instancegroups.go:407] Draining the node: "node-2b.local". I1206 14:25:05.354862 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.354901 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.356088 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.356138 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.361276 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.361331 35813 instancegroups.go:407] Draining the node: "node-2c.local". I1206 14:25:05.362207 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.362248 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.363460 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.363494 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.368732 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.368766 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I1206 14:25:05.368896 35813 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I1206 14:25:05.369376 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.369410 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.370600 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.370642 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.370663 35813 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I1206 14:25:05.371013 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.371046 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.372248 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.372284 35813 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1206 14:25:05.372306 35813 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I1206 14:25:05.372559 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.372591 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.372606 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.372625 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1206 14:25:05.372640 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I1206 14:25:05.372787 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.372813 35813 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1206 14:25:05.372828 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.372846 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1206 14:25:05.372860 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I1206 14:25:05.373073 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.373103 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.374298 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.374340 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.379612 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.379679 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.379704 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.379717 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.380563 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.381333 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.381372 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.382588 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.382636 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.382658 35813 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I1206 14:25:05.382937 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.382969 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.384174 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.384214 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.389461 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.389528 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.389553 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.389566 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.390272 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.391053 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.391091 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.392288 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.392331 35813 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1206 14:25:05.392353 35813 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I1206 14:25:05.392663 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.392692 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.392706 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.393749 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.394518 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.394556 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.395760 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.395797 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.401054 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.401105 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.401969 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.402008 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.403216 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.403252 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.408499 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.408562 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.409420 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.409459 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.410658 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.410694 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.416241 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.416277 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I1206 14:25:05.416550 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.416583 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.416601 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.416621 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1206 14:25:05.416635 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I1206 14:25:05.416857 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.416887 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.416907 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.416927 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1206 14:25:05.416942 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I1206 14:25:05.417188 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.417217 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.417233 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.417251 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I1206 14:25:05.417265 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I1206 14:25:05.417417 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.417445 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.417459 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.418564 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.419523 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.419562 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.420767 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.420814 35813 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E1206 14:25:05.420835 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.420856 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I1206 14:25:05.420871 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I1206 14:25:05.421077 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.421109 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.421122 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.422722 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.423625 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.423664 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.424873 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.424915 35813 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E1206 14:25:05.424955 35813 instancegroups.go:482] Cluster did not validate within 0s E1206 14:25:05.424978 35813 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I1206 14:25:05.424994 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I1206 14:25:05.425300 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.425332 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.426545 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.426595 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.427832 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.433109 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.434314 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.439594 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.440776 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.446111 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.446202 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.446246 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.446266 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.447265 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.448848 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.448946 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.450274 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.450335 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.456099 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.456153 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.456912 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.456945 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.458153 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.458213 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.463405 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.463455 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.463474 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.463484 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.464622 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.465472 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.465508 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.466720 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.466774 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.472039 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.472080 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.472799 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.472827 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.474041 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.474095 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.479422 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.479485 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.480425 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.480462 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.481680 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.481733 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.487039 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.487071 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.487100 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.487115 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.488243 35813 instancegroups.go:407] Draining the node: "node-2a.local". I1206 14:25:05.489192 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.489229 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.490437 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.490488 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.495773 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.495833 35813 instancegroups.go:407] Draining the node: "node-2b.local". I1206 14:25:05.496828 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.496871 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.498058 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.498117 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.503361 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.503424 35813 instancegroups.go:407] Draining the node: "node-2c.local". I1206 14:25:05.504368 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.504404 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.505617 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.505670 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.510965 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.511003 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.09s) === RUN TestRollingUpdateValidatesAfterBastion I1206 14:25:05.511397 35813 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I1206 14:25:05.511429 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.512647 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.512695 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.513880 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.515052 35813 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I1206 14:25:05.516237 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.521501 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.521581 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.521624 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.521642 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.522625 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.523539 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.523583 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.524809 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.524869 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.530126 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.530183 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.531127 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.531171 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.532354 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.532386 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.537596 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.537639 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.537667 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.537682 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.538774 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.539482 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.539516 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.540733 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.540770 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.546027 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.546093 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.547577 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.547619 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.548763 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.548796 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.554056 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.554110 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.554955 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.554995 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.556216 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.556253 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.561511 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.561545 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.561567 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.561582 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.562598 35813 instancegroups.go:407] Draining the node: "node-2a.local". I1206 14:25:05.563397 35813 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I1206 14:25:05.563434 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.564651 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.564688 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.569930 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.569984 35813 instancegroups.go:407] Draining the node: "node-2b.local". I1206 14:25:05.570809 35813 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I1206 14:25:05.570845 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.572045 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.572083 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.577347 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.577400 35813 instancegroups.go:407] Draining the node: "node-2c.local". I1206 14:25:05.578244 35813 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I1206 14:25:05.578301 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.579515 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.579553 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.584799 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.584836 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I1206 14:25:05.585546 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.585573 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.585585 35813 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1206 14:25:05.586335 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.587176 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.587212 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.588410 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.588446 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.593730 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.593782 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.594617 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.594653 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.595883 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.595928 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.601224 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.601258 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I1206 14:25:05.601501 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.601529 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.601541 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.602305 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:05.603097 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:05.603134 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.604351 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.604388 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.609630 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.609681 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:05.610517 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:05.610553 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.611749 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.611785 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.617053 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.617087 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I1206 14:25:05.617399 35813 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W1206 14:25:05.617447 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1206 14:25:05.617469 35813 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W1206 14:25:05.617486 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1206 14:25:05.617507 35813 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W1206 14:25:05.617519 35813 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I1206 14:25:05.617539 35813 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I1206 14:25:05.617555 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I1206 14:25:05.617826 35813 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I1206 14:25:05.617882 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:05.619107 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.619144 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.624362 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.624402 35813 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I1206 14:25:05.624462 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.624486 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.624501 35813 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I1206 14:25:05.625313 35813 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I1206 14:25:05.625347 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.625367 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.625381 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:05.626255 35813 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1206 14:25:05.626320 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:05.627541 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.627576 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.632787 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.632828 35813 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I1206 14:25:05.632844 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.632863 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.632878 35813 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I1206 14:25:05.633847 35813 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I1206 14:25:05.633906 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:05.635097 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.635132 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:05.640422 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.640476 35813 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I1206 14:25:05.640499 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I1206 14:25:05.640946 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.641000 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.641024 35813 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I1206 14:25:05.643585 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.644392 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.644442 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.645649 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.645695 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.645739 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.645796 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.647253 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.647556 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.647599 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.648790 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.648902 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.749465 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.749566 35813 instancegroups.go:407] Draining the node: "node-1e.local". I1206 14:25:05.749585 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:05.751260 35813 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1206 14:25:05.751688 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:05.751745 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.752998 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.753052 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.753098 35813 instancegroups.go:407] Draining the node: "node-1f.local". I1206 14:25:05.754257 35813 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1206 14:25:05.754306 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.755497 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.755575 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.856626 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.856730 35813 instancegroups.go:407] Draining the node: "node-1g.local". I1206 14:25:05.857723 35813 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1206 14:25:05.857779 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.858975 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.859023 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.859043 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I1206 14:25:05.859330 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.859364 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.859380 35813 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1206 14:25:05.861245 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:05.861315 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:05.862748 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:05.863110 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:05.863163 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.864415 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.864487 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.965026 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.965084 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:05.965157 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:05.966530 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:05.966855 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:05.966905 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.968132 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.968177 35813 instancegroups.go:510] Cluster validated. I1206 14:25:05.968220 35813 instancegroups.go:407] Draining the node: "node-1e.local". I1206 14:25:05.969112 35813 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1206 14:25:05.969161 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:05.970375 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:05.970444 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.071009 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.071093 35813 instancegroups.go:407] Draining the node: "node-1f.local". I1206 14:25:06.072164 35813 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1206 14:25:06.072221 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.073465 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.073534 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.073556 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I1206 14:25:06.073894 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.073930 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.073945 35813 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I1206 14:25:06.076280 35813 instancegroups.go:407] Draining the node: "master-1a.local". I1206 14:25:06.077064 35813 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I1206 14:25:06.077113 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.078347 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.078395 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.078438 35813 instancegroups.go:407] Draining the node: "master-1c.local". I1206 14:25:06.078518 35813 instancegroups.go:407] Draining the node: "master-1b.local". I1206 14:25:06.079759 35813 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I1206 14:25:06.080050 35813 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I1206 14:25:06.080094 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.081308 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.081375 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.181975 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.182029 35813 instancegroups.go:407] Draining the node: "master-1e.local". I1206 14:25:06.182116 35813 instancegroups.go:407] Draining the node: "master-1d.local". I1206 14:25:06.183453 35813 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I1206 14:25:06.183757 35813 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I1206 14:25:06.183804 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.185038 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.185084 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.185127 35813 instancegroups.go:407] Draining the node: "master-1f.local". I1206 14:25:06.186955 35813 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I1206 14:25:06.187018 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.188682 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.188776 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.289009 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.289065 35813 instancegroups.go:407] Draining the node: "master-1g.local". I1206 14:25:06.289983 35813 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I1206 14:25:06.290030 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.291260 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.291326 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.291349 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.22s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I1206 14:25:06.291648 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.291688 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.291703 35813 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1206 14:25:06.293560 35813 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1206 14:25:06.293637 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.294857 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.294905 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.294921 35813 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1206 14:25:06.295004 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.296617 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.296663 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.296710 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.296786 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.298169 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.298583 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.298637 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.299853 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.299916 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.400507 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.400560 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:06.400587 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:06.401960 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:06.402266 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:06.402310 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.403535 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.403581 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.403626 35813 instancegroups.go:407] Draining the node: "node-1e.local". I1206 14:25:06.404637 35813 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1206 14:25:06.404690 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.405901 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.405974 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.506674 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.506728 35813 instancegroups.go:407] Draining the node: "node-1f.local". I1206 14:25:06.507701 35813 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1206 14:25:06.507753 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.508970 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.509014 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.509035 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I1206 14:25:06.509357 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.509394 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.509408 35813 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I1206 14:25:06.511552 35813 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I1206 14:25:06.511599 35813 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I1206 14:25:06.511668 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.512894 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.512941 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.512989 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.513063 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.514462 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.514759 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.514806 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.516023 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.516096 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.616578 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.616635 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:06.616666 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:06.618073 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:06.618398 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:06.618444 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.619658 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.619704 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.619747 35813 instancegroups.go:407] Draining the node: "node-1e.local". I1206 14:25:06.620652 35813 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1206 14:25:06.620704 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.621907 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.621988 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.722793 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.722859 35813 instancegroups.go:407] Draining the node: "node-1f.local". I1206 14:25:06.723772 35813 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1206 14:25:06.723825 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.725020 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.725064 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.725084 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I1206 14:25:06.725315 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.725348 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.725362 35813 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1206 14:25:06.726123 35813 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1206 14:25:06.726157 35813 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I1206 14:25:06.726195 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.727396 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.727432 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:06.732677 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.732733 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.732755 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.734305 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.734370 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.734642 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.734683 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.735582 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.735618 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:06.740947 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.740982 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.02s) === RUN TestRollingUpdateDetachFails I1206 14:25:06.741212 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.741240 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.741253 35813 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I1206 14:25:06.741997 35813 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E1206 14:25:06.742059 35813 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I1206 14:25:06.742073 35813 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E1206 14:25:06.742113 35813 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I1206 14:25:06.742146 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.743341 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.743376 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:06.748612 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.748665 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.748687 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.750046 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.750085 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.750494 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.750548 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.751731 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.751781 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:06.757079 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.757127 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.02s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I1206 14:25:06.757398 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.757430 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.757443 35813 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I1206 14:25:06.758705 35813 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I1206 14:25:06.758769 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.759967 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.760017 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.760032 35813 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I1206 14:25:06.760086 35813 instancegroups.go:179] waiting for 1ms after detaching instance I1206 14:25:06.761288 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.761331 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.761377 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:06.761407 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.761463 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.763262 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:06.763316 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.763620 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.763669 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.763925 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.763967 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.764528 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.865183 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.865252 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:06.866130 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:06.866182 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.867386 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.867437 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.867457 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I1206 14:25:06.867692 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.867725 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.867740 35813 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I1206 14:25:06.869859 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:06.870619 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:06.870664 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.871858 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.871903 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.871947 35813 instancegroups.go:407] Draining the node: "node-1e.local". I1206 14:25:06.872022 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:06.873491 35813 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I1206 14:25:06.873907 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:06.873983 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.875177 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.875258 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.976131 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.976202 35813 instancegroups.go:407] Draining the node: "node-1g.local". I1206 14:25:06.976274 35813 instancegroups.go:407] Draining the node: "node-1f.local". I1206 14:25:06.977646 35813 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I1206 14:25:06.977960 35813 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I1206 14:25:06.978007 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.979262 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.979342 35813 instancegroups.go:510] Cluster validated. I1206 14:25:06.979401 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:06.980617 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:06.980670 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:06.981875 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:06.981949 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:07.082766 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.082850 35813 instancegroups.go:407] Draining the node: "node-1d.local". I1206 14:25:07.083932 35813 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I1206 14:25:07.083989 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:07.085213 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.085275 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.085297 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.22s) === RUN TestRollingUpdateOnlyWarmPoolNodes I1206 14:25:07.085601 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.085628 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.085644 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1206 14:25:07.085666 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1206 14:25:07.085682 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1206 14:25:07.085703 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I1206 14:25:07.085865 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.085890 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.085904 35813 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I1206 14:25:07.087031 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I1206 14:25:07.087061 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I1206 14:25:07.087077 35813 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I1206 14:25:07.087120 35813 instancegroups.go:407] Draining the node: "node-1a.local". I1206 14:25:07.087931 35813 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I1206 14:25:07.087966 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:07.089174 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.089213 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:07.094485 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.094543 35813 instancegroups.go:407] Draining the node: "node-1b.local". I1206 14:25:07.095383 35813 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I1206 14:25:07.095420 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:07.096600 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.096652 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:07.101969 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.102033 35813 instancegroups.go:407] Draining the node: "node-1c.local". I1206 14:25:07.103210 35813 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I1206 14:25:07.103278 35813 instancegroups.go:443] waiting for 1ms after terminating instance I1206 14:25:07.104447 35813 instancegroups.go:477] Validating the cluster. I1206 14:25:07.104479 35813 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I1206 14:25:07.109689 35813 instancegroups.go:510] Cluster validated. I1206 14:25:07.109726 35813 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.007s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.033s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.026s === RUN TestParse E1206 14:24:40.543443 31575 versions.go:96] unable to parse Kubernetes version "" E1206 14:24:40.543524 31575 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.037s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.056s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.28s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.06s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.07s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.04s) PASS ok k8s.io/kops/pkg/kubeconfig 0.426s ? k8s.io/kops/pkg/kubemanifest [no test files] === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.092s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.081s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.121s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I1206 14:24:47.496600 33599 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.150s ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] === RUN Test_RunEtcdManagerBuilder I1206 14:24:51.393426 34554 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W1206 14:24:51.460305 34554 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W1206 14:24:51.462170 34554 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.07s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.229s ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] === RUN Test_RunKubeApiserverBuilder I1206 14:24:51.917161 34915 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.01s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.01s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.142s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.03s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.125s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.065s ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.157s === RUN TestServerGroupBuilder I1206 14:24:52.192261 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.210325 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.224814 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.238612 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.304211 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.319042 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.331728 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.345530 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.352699 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.457954 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.465030 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.472332 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.480787 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.487772 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.493497 34993 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.31s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.456s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.068s ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.012s ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.049s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.71s) --- PASS: TestIssueCert/ca (0.50s) --- PASS: TestIssueCert/client (0.49s) --- PASS: TestIssueCert/clientOneYear (0.72s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.749s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.014s ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] === RUN TestAddUntaggedRouteTables I1206 14:24:52.786719 35181 routetable.go:53] DescribeRouteTables: { } I1206 14:24:52.786847 35181 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W1206 14:24:52.786866 35181 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I1206 14:24:52.786992 35181 iaminstanceprofile.go:193] ListInstanceProfiles: { } W1206 14:24:52.787069 35181 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I1206 14:24:52.787152 35181 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I1206 14:24:52.787251 35181 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I1206 14:24:52.787345 35181 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I1206 14:24:52.787457 35181 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I1206 14:24:52.787588 35181 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I1206 14:24:52.787626 35181 volumes.go:128] DescribeVolumes: { } W1206 14:24:52.787668 35181 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I1206 14:24:52.787827 35181 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:52.788045 35181 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:52.788090 35181 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:52.788165 35181 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:52.788232 35181 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:52.788294 35181 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:52.788321 35181 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:52.788356 35181 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:52.788387 35181 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:52.788415 35181 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I1206 14:24:52.788454 35181 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I1206 14:24:52.788483 35181 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I1206 14:24:52.788523 35181 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I1206 14:24:52.788552 35181 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I1206 14:24:52.788602 35181 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I1206 14:24:52.788644 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I1206 14:24:52.788686 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I1206 14:24:52.788717 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I1206 14:24:52.788746 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I1206 14:24:52.788779 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I1206 14:24:52.788809 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I1206 14:24:52.788841 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I1206 14:24:52.788870 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I1206 14:24:52.788898 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I1206 14:24:52.788929 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I1206 14:24:52.788963 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I1206 14:24:52.788988 35181 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.064s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.076s ? k8s.io/kops/pkg/resources/digitalocean [no test files] === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.215s ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.047s ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.037s ? k8s.io/kops/pkg/try [no test files] === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.016s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.039s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.009s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.01s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.113s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.069s ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.011s ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] === RUN TestRemovesDuplicateGuardedBlocks W1206 14:24:45.237338 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237399 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237411 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237422 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237433 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237443 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237453 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237464 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237474 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237484 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237495 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237506 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237523 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237534 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.237565 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.237581 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240023 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240058 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240132 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240149 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240213 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240229 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240293 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240315 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240385 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240402 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240488 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240505 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240573 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240592 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240666 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240685 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240751 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240769 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240834 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240850 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.240938 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.240956 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241026 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241043 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241110 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241126 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241192 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241208 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241276 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241290 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241354 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241371 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241434 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241448 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241508 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241523 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241595 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241623 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241741 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241759 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241819 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241834 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241898 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241913 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.241971 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.241988 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242054 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242070 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242134 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242151 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242230 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242246 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242313 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242329 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242396 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242412 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242476 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242493 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242559 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242576 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242648 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242665 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242743 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242769 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242837 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242853 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242916 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.242932 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.242995 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243011 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243077 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243092 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243175 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243192 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243255 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243270 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243329 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243340 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243411 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243421 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243503 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243519 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243563 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243573 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243627 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243638 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243687 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243705 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243767 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243783 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243841 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243857 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243905 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243915 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.243987 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.243998 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244043 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244057 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244097 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244107 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244157 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244695 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244752 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244763 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244811 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244820 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244860 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244870 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.244924 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.244933 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245002 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245016 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245067 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245083 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245146 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245160 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245216 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245229 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245284 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245298 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245348 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245424 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245521 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245541 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245646 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245689 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245787 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245829 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.245915 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.245941 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246000 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246014 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246073 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246086 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246142 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246155 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246231 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246245 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246299 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246312 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246368 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246382 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246444 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246458 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246522 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246541 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246599 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246614 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246674 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246690 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246739 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246773 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246833 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246849 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246901 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246916 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.246967 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.246980 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247035 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247049 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247102 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247116 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247175 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247192 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247250 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247265 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247351 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247365 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247423 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247438 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247493 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247508 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247561 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247576 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247626 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247640 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247690 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247704 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247762 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247778 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247830 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247843 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247895 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247909 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.247960 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.247976 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248025 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248038 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248089 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248103 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248163 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248179 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248228 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248241 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248308 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248323 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.248374 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.248387 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W1206 14:24:45.248957 33152 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1206 14:24:45.248970 33152 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1206 14:24:45.248981 33152 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1206 14:24:45.248991 33152 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W1206 14:24:45.249000 33152 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1206 14:24:45.249009 33152 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1206 14:24:45.249017 33152 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W1206 14:24:45.249045 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249057 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249067 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.249078 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249088 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.249098 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249108 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.249141 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249153 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.249164 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249174 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249184 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249195 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249205 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.249215 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.249226 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258375 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258406 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258490 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258509 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258564 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258599 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258662 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258676 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258763 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258777 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258832 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258846 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258905 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.258919 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.258987 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259001 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259059 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259074 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259132 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259146 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259195 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259230 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259290 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259305 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259373 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259388 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259441 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259457 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259507 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259522 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259573 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259587 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259641 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259655 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259714 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259727 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259781 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259794 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259845 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259879 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.259936 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.259950 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260023 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260038 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260094 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260109 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260163 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260197 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260256 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260269 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260324 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260340 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260402 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260418 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260467 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260484 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260566 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260581 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260631 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260685 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260745 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260759 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260817 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260832 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260886 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260903 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.260962 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.260977 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261027 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261059 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261119 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261135 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261185 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261202 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261251 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261265 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261325 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261340 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261395 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261409 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261465 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261479 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261533 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261547 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261618 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261634 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261691 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261705 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261756 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261789 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261847 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261860 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.261914 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.261946 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262007 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262020 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262071 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262103 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262163 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262177 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262229 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262246 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262304 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262321 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262377 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262411 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262487 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262504 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262561 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262575 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262630 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262644 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262693 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262726 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262783 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262797 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262852 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262866 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262922 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.262936 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.262993 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263006 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263061 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263075 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263127 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263145 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263208 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263224 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263276 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263309 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263372 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263386 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263439 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263454 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263504 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263519 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263570 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263584 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263638 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263652 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263710 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263723 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263778 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263791 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263845 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263860 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263911 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263927 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.263975 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.263989 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264041 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264054 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264110 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264124 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264202 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264216 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264272 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264285 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264344 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264357 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264420 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264435 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264491 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264504 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264573 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264596 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264665 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264679 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264746 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264760 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264826 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264841 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.264916 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.264931 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265001 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265017 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265079 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265091 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265153 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265165 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265247 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265263 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265328 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265342 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265407 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265421 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265485 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265499 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265631 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265650 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265714 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265728 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265790 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265804 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265869 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265883 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W1206 14:24:45.265947 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W1206 14:24:45.265961 33152 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.048s ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.074s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.104s ? k8s.io/kops/tests/fuzz [no test files] === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.04s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.175s ? k8s.io/kops/tests/integration/channel/simple [no test files] === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.10s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.04s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.04s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.02s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.02s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.02s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.02s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.02s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.01s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.289s ? k8s.io/kops/upup/models [no test files] === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I1206 14:24:50.952095 34404 files.go:57] Writing file "/tmp/TestWriteFile2532938584/001/SubDir/test1.tmp" I1206 14:24:50.952287 34404 files.go:113] Changing file mode for "/tmp/TestWriteFile2532938584/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.061s === RUN TestBootstrapChannelBuilder_BuildTasks I1206 14:24:57.455008 35660 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:57.455257 35660 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:57.455307 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:57.455346 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:57.455396 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:57.455435 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:57.455450 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:57.455487 35660 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:57.455515 35660 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:57.464858 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.514644 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.632587 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.699450 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.762944 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.823675 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.879945 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:57.941520 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:58.033741 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:58.445289 35660 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.07s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I1206 14:24:58.523985 35660 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:58.524144 35660 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:58.524215 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:58.524248 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:58.524279 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:58.524306 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:58.524324 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:58.524346 35660 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:58.524365 35660 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:58.524385 35660 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I1206 14:24:58.525585 35660 zones.go:93] ListHostedZonesPages { } I1206 14:24:58.617438 35660 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.09s) === RUN TestBootstrapChannelBuilder_AWSCloudController I1206 14:24:58.617741 35660 featureflag.go:160] FeatureFlag "ImageDigest"=false I1206 14:24:58.617999 35660 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:58.618095 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I1206 14:24:58.618151 35660 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I1206 14:24:58.618202 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I1206 14:24:58.618243 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I1206 14:24:58.618291 35660 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I1206 14:24:58.618330 35660 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I1206 14:24:58.618361 35660 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I1206 14:24:58.620253 35660 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.07s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.12s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W1206 14:24:58.967492 35660 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.01s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I1206 14:24:59.025181 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025223 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025244 35660 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1206 14:24:59.025278 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1206 14:24:59.025298 35660 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I1206 14:24:59.025333 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I1206 14:24:59.025368 35660 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I1206 14:24:59.025401 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025424 35660 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1206 14:24:59.025455 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025476 35660 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1206 14:24:59.025512 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025555 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025599 35660 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I1206 14:24:59.025620 35660 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I1206 14:24:59.025637 35660 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I1206 14:24:59.025656 35660 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I1206 14:24:59.025672 35660 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I1206 14:24:59.025691 35660 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I1206 14:24:59.025709 35660 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I1206 14:24:59.025730 35660 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I1206 14:24:59.027551 35660 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.675s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I1206 14:24:55.566633 35575 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2230964645/001 I1206 14:24:55.569942 35575 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2230964645/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.00s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I1206 14:24:55.574472 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1206 14:24:55.574561 35575 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1206 14:24:55.574640 35575 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I1206 14:24:55.574861 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.574990 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.575062 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.575079 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.575215 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.575255 35575 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1206 14:24:55.575288 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.575374 35575 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1206 14:24:55.575451 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1206 14:24:55.575584 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.575635 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.575682 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.575697 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.575727 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.575766 35575 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1206 14:24:55.575818 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I1206 14:24:55.576120 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.576167 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.576263 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.576303 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.576326 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.576348 35575 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1206 14:24:55.576447 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.576633 35575 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1206 14:24:55.576685 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.576707 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.576731 35575 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1206 14:24:55.576784 35575 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.576924 35575 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I1206 14:24:55.576974 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.576995 35575 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" },{ Key: "AssociatedElasticIp", Value: "192.0.2.1" }] } I1206 14:24:55.577033 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1206 14:24:55.577175 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.577220 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.577264 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.577280 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.577305 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.577327 35575 executor.go:111] Tasks: 1 done / 3 total; 1 can run I1206 14:24:55.577361 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.577412 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.577436 35575 executor.go:111] Tasks: 2 done / 3 total; 1 can run I1206 14:24:55.577469 35575 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.577512 35575 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I1206 14:24:55.577548 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I1206 14:24:55.577583 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I1206 14:24:55.577719 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I1206 14:24:55.577761 35575 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I1206 14:24:55.577783 35575 internetgateways.go:66] CreateInternetGateway: { } I1206 14:24:55.577798 35575 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I1206 14:24:55.577921 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.577962 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.577999 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.578014 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.578075 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.578101 35575 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1206 14:24:55.578131 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.578184 35575 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1206 14:24:55.578252 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1206 14:24:55.578424 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.578472 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.578517 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.578533 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.578572 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.578611 35575 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I1206 14:24:55.578670 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I1206 14:24:55.580472 35575 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3459210096/001 I1206 14:24:55.581872 35575 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender3459210096/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I1206 14:24:55.582423 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.582472 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.582564 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.582602 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.582625 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.582651 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.582711 35575 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1206 14:24:55.582903 35575 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I1206 14:24:55.582944 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I1206 14:24:55.582975 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1206 14:24:55.583103 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.583149 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.583191 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.583208 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.583234 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.583257 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.583293 35575 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I1206 14:24:55.583358 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I1206 14:24:55.583705 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.583746 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.583824 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.583863 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.583884 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.583907 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.583943 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.584032 35575 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1206 14:24:55.584069 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.584090 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.584111 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1206 14:24:55.584246 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.584287 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.584328 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.584343 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.584366 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.584389 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.584433 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.584489 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.584512 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I1206 14:24:55.584792 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.584835 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.584928 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.584979 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.585006 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.585034 35575 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1206 14:24:55.585104 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.585238 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.585392 35575 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1206 14:24:55.585403 35575 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:24:55.585465 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.585503 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.585526 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1206 14:24:55.585682 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.585732 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.585779 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.585800 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.585828 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.585852 35575 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1206 14:24:55.585902 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.585962 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.586015 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.586040 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I1206 14:24:55.586341 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.586380 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.586452 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.586489 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.586508 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.586527 35575 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1206 14:24:55.586556 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.586661 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.586699 35575 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I1206 14:24:55.586739 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.586775 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.586802 35575 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I1206 14:24:55.586837 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run I1206 14:24:55.587019 35575 executor.go:111] Tasks: 0 done / 3 total; 1 can run I1206 14:24:55.587071 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.587125 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.587144 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.587175 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.587207 35575 executor.go:111] Tasks: 1 done / 3 total; 2 can run I1206 14:24:55.587263 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.587324 35575 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I1206 14:24:55.587428 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.587465 35575 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I1206 14:24:55.587634 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I1206 14:24:55.587717 35575 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I1206 14:24:55.587877 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.587929 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.587980 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.588001 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.588099 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.588135 35575 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1206 14:24:55.588176 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.588222 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.588361 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I1206 14:24:55.588394 35575 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I1206 14:24:55.588434 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run I1206 14:24:55.588603 35575 executor.go:111] Tasks: 0 done / 2 total; 1 can run I1206 14:24:55.588655 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.588701 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.588721 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.588758 35575 executor.go:111] Tasks: 1 done / 2 total; 1 can run I1206 14:24:55.588799 35575 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I1206 14:24:55.588874 35575 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I1206 14:24:55.589159 35575 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.589210 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.589314 35575 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I1206 14:24:55.589361 35575 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I1206 14:24:55.589393 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.589424 35575 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.589558 35575 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.589608 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I1206 14:24:55.589671 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.589693 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.589725 35575 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I1206 14:24:55.589755 35575 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I1206 14:24:55.589989 35575 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I1206 14:24:55.590142 35575 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.590194 35575 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I1206 14:24:55.590248 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I1206 14:24:55.590268 35575 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I1206 14:24:55.590308 35575 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.080s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.045s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.082s === RUN TestDiskRenderAzure I1206 14:24:51.621617 34778 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I1206 14:24:51.622008 34778 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I1206 14:24:51.622337 34778 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I1206 14:24:51.622925 34778 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I1206 14:24:51.623057 34778 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I1206 14:24:51.623291 34778 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I1206 14:24:51.623516 34778 resourcegroup.go:103] Creating a new Resource Group with name: rg I1206 14:24:51.623532 34778 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I1206 14:24:51.623727 34778 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I1206 14:24:51.624270 34778 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I1206 14:24:51.624527 34778 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I1206 14:24:51.624764 34778 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I1206 14:24:51.625078 34778 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I1206 14:24:51.625474 34778 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.064s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.142s ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestProjectIAMBinding I1206 14:24:55.203240 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.204037 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.204250 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.204825 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.205042 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.205255 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I1206 14:24:55.205555 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.205702 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.205876 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.206061 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.206240 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.206319 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I1206 14:24:55.206617 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.206957 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.207129 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.207623 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run I1206 14:24:55.207804 35544 executor.go:111] Tasks: 0 done / 1 total; 1 can run I1206 14:24:55.207990 35544 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.070s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.037s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.031s ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.071s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.048s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.069s ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I1206 14:24:57.126959 35645 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I1206 14:24:57.127015 35645 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I1206 14:24:57.127026 35645 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I1206 14:24:57.127037 35645 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I1206 14:24:57.127047 35645 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I1206 14:24:57.127064 35645 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I1206 14:24:57.127078 35645 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W1206 14:24:57.127516 35645 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.058s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.039s ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.019s ? k8s.io/kops/util/pkg/env [no test files] === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.018s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.005s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.005s ? k8s.io/kops/util/pkg/mirrors [no test files] === RUN TestGetProxyEnvVars W1206 14:24:53.388629 35382 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.023s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.013s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.018s ? k8s.io/kops/util/pkg/tables [no test files] === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.032s ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.022s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W1206 14:24:58.202072 35771 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.064s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 133.6 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r0.apk >>> kops: Build complete at Tue, 06 Dec 2022 14:25:46 +0000 elapsed time 0h 3m 20s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20221206.142228) (2/3) Purging kubernetes (1.25.0-r3) (3/3) Purging go (1.19.3-r0) Executing busybox-1.35.0-r29.trigger OK: 393 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...