>>> kops: Building testing/kops 1.25.3-r2 (using abuild 3.10.0-r0) started Wed, 08 Feb 2023 04:07:37 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.26.1-r1) (2/3) Installing go (1.20-r0) (3/3) Installing .makedepends-kops (20230208.040739) Executing busybox-1.36.0-r3.trigger OK: 562 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 74 31.1M 74 23.0M 0 0 32.7M 0 --:--:-- --:--:-- --:--:-- 32.7M 100 31.1M 100 31.1M 0 0 37.4M 0 --:--:-- --:--:-- --:--:-- 37.4M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading k8s.io/apimachinery v0.25.4 go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading google.golang.org/api v0.103.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading k8s.io/client-go v0.25.4 go: downloading github.com/spf13/viper v1.14.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/api v0.25.4 go: downloading k8s.io/klog v1.0.0 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/stretchr/testify v1.8.1 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading golang.org/x/crypto v0.3.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading google.golang.org/appengine v1.6.7 go: downloading golang.org/x/net v0.2.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/term v0.2.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/kubectl v0.25.4 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading k8s.io/component-base v0.25.4 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading golang.org/x/sys v0.2.0 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading github.com/golang/protobuf v1.5.2 go: downloading google.golang.org/protobuf v1.28.1 go: downloading golang.org/x/sync v0.1.0 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading golang.org/x/text v0.4.0 go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.4 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading google.golang.org/grpc v1.50.1 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading go.opencensus.io v0.24.0 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/moby/locker v1.0.1 go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0208 04:09:54.300361 2354 addon.go:233] addon test wants to update all nodes I0208 04:09:54.303124 2354 addon.go:233] addon test wants to update worker nodes I0208 04:09:54.303596 2354 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0208 04:09:54.304292 2354 addon.go:266] installing PKI for "test" I0208 04:09:55.054052 2354 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.20s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.266s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.077s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestCreateClusterMinimal I0208 04:09:56.274704 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.275113 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.275184 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.275256 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.275346 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.275411 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.275451 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.275505 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.275558 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.275726 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.275802 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.357616 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.357927 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.358522 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.378734 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.378912 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.378947 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.378977 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.379026 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.379055 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.379074 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.379103 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.379129 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.379196 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.379222 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.421858 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.422032 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.422306 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.427873 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.428042 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.428074 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.428101 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.428135 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.428159 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.428177 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.428198 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.428214 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.428269 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.428292 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.475228 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.475395 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.475693 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.481254 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.481420 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.481452 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.481483 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.481515 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.481541 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.481559 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.481582 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.481598 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.481657 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.481681 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.517976 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.518176 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.518454 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.524056 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.524222 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.524254 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.524282 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.524316 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.524344 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.524361 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.524384 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.524401 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.524456 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.524480 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.555002 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.555244 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.555718 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.562525 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.562711 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.562742 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.562775 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.562819 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.562847 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.562863 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.562885 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.562900 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.562957 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.562991 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.645178 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.645395 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.645740 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.651732 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.651900 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.651940 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.651978 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.652014 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.652042 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.652060 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.652085 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.652104 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.652197 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.652226 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.682232 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.682431 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.682758 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:56.688684 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.688854 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.688894 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.688931 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.688969 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.688998 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.689040 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.689072 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.689091 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.689155 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.689183 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.717555 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.717653 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.717958 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.45s) === RUN TestCreateClusterHetzner I0208 04:09:56.724049 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.724251 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.724294 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.724328 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.724363 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.724393 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.724414 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.724437 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.724456 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.724533 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.724561 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0208 04:09:56.860095 3399 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0208 04:09:56.860151 3399 new_cluster.go:1279] Cloud Provider ID = hetzner I0208 04:09:56.870161 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.870341 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.870462 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.870560 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.870620 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.870705 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.870775 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.870855 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.870918 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.871042 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.871119 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.921521 3399 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.21s) === RUN TestCreateClusterOverride I0208 04:09:56.931768 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.932023 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.932077 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.932130 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.932186 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.932235 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.932267 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.932307 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.932346 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.932473 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.932520 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.975136 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:56.975665 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:56.976113 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.05s) === RUN TestCreateClusterComplex I0208 04:09:56.984850 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:56.985041 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:56.985091 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:56.985142 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:56.985193 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:56.985237 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:56.985266 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:56.985306 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:56.985333 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:56.985420 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:56.985455 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.089874 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.090048 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.090366 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.11s) === RUN TestCreateClusterHA I0208 04:09:57.096120 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.096336 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.096377 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.096411 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.096450 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.096480 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.096500 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.096525 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.096546 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.096616 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.096645 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.120894 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.121442 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.121464 3399 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0208 04:09:57.121481 3399 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0208 04:09:57.121962 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:57.132641 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.132794 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.132837 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.132870 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.132905 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.132933 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.132951 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.132976 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.132995 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.133058 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.133088 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.176880 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.177242 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.177258 3399 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0208 04:09:57.177268 3399 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0208 04:09:57.177626 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.09s) === RUN TestCreateClusterHAGCE I0208 04:09:57.186686 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.186860 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.186897 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.186928 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.186961 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.186986 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.187002 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.187024 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.187040 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.187111 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.187139 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.238339 3399 new_cluster.go:1279] Cloud Provider ID = gce I0208 04:09:57.238471 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.06s) === RUN TestCreateClusterGCE I0208 04:09:57.246802 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.246970 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.247003 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.247030 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.247059 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.247085 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.247102 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.247125 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.247142 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.247208 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.247234 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.303378 3399 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0208 04:09:57.303446 3399 new_cluster.go:1279] Cloud Provider ID = gce I0208 04:09:57.303514 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.07s) === RUN TestCreateClusterHASharedZone I0208 04:09:57.312894 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.313103 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.313136 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.313180 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.313229 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.313255 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.313272 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.313293 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.313310 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.313375 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.313399 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0208 04:09:57.347617 3399 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0208 04:09:57.347670 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.347910 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.348195 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.04s) === RUN TestCreateClusterHASharedZones I0208 04:09:57.355276 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.355436 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.355468 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.355495 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.355526 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.355551 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.355568 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.355591 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.355607 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.355695 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.355722 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0208 04:09:57.380546 3399 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0208 04:09:57.380616 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.380958 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.380974 3399 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0208 04:09:57.381294 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.04s) === RUN TestCreateClusterPrivate I0208 04:09:57.390371 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.390547 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.390578 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.390604 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.390635 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.390661 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.390678 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.390699 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.390715 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.390780 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.390804 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.480259 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.480485 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.480502 3399 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0208 04:09:57.480869 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:57.480905 3399 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.10s) === RUN TestCreateClusterPrivateGCE I0208 04:09:57.488369 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.488544 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.488585 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.488618 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.488653 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.488681 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.488697 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.488719 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.488739 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.488811 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.488837 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.530265 3399 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0208 04:09:57.530351 3399 new_cluster.go:1279] Cloud Provider ID = gce I0208 04:09:57.530435 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.05s) === RUN TestCreateClusterWithNGWSpecified I0208 04:09:57.538449 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.538623 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.538663 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.538692 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.538727 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.538755 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.538774 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.538800 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.538820 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.538892 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.538918 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.577289 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.577502 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.577518 3399 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0208 04:09:57.577871 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:57.577902 3399 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.05s) === RUN TestCreateClusterWithINGWSpecified I0208 04:09:57.587666 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.587981 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.588026 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.588059 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.588094 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.588120 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.588136 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.588157 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.588172 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.588242 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.588269 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.643838 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.644043 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.644059 3399 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0208 04:09:57.644363 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:57.644392 3399 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.06s) === RUN TestCreateClusterSharedVPC I0208 04:09:57.650511 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.650681 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.650717 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.650744 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.650775 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.650802 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.650818 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.650840 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.650858 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.650923 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.650948 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.731671 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.731850 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.731898 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.731938 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.731957 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.732002 3399 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0208 04:09:57.732266 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.09s) === RUN TestCreateClusterSharedSubnets I0208 04:09:57.737374 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.737534 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.737565 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.737593 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.737622 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.737651 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.737670 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.737692 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.737707 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.737771 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.737796 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.762023 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.762073 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.762143 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.762272 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.762294 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.762317 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.762342 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.762646 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.03s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0208 04:09:57.767862 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.768021 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.768052 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.768078 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.768109 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.768134 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.768151 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.768172 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.768187 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.768254 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.768280 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.820520 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:09:57.820600 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.820623 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.820690 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.820818 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.820841 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.820867 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.820882 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.821192 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.06s) === RUN TestCreateClusterPrivateSharedSubnets I0208 04:09:57.826415 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.826574 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.826604 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.826630 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.826659 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.826684 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.826700 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.826722 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.826739 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.826832 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.826860 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.925959 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.926023 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.926130 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.926170 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.926223 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.926464 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.926505 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.926549 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:09:57.926578 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0208 04:09:57.927077 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.11s) === RUN TestCreateClusterIPv6 I0208 04:09:57.937701 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.937924 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.937972 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.938016 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.938064 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.938102 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.938130 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.938166 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.938191 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.938290 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.938330 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.983359 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:57.983624 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:57.984305 3399 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestCreateClusterDifferentAMIs I0208 04:09:57.990691 3399 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0208 04:09:57.990708 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:57.990870 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:57.990904 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:57.990936 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:57.990970 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:57.990998 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:57.991015 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:57.991038 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:57.991056 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:57.991125 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:57.991151 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:58.037299 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:58.037390 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:58.037404 3399 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0208 04:09:58.037738 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:58.045184 3399 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.05s) === RUN TestCreateClusterKarpenter I0208 04:09:58.045317 3399 featureflag.go:160] FeatureFlag "Karpenter"=true I0208 04:09:58.045330 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:58.045499 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:58.045552 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:58.045584 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:58.045619 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:58.045647 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:58.045664 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:58.045698 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:58.045716 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:58.045790 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:58.045819 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0208 04:09:58.088180 3399 new_cluster.go:1279] Cloud Provider ID = aws I0208 04:09:58.088358 3399 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0208 04:09:58.088373 3399 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0208 04:09:58.088382 3399 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0208 04:09:58.088735 3399 zones.go:93] ListHostedZonesPages { } I0208 04:09:58.096651 3399 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.05s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0208 04:09:58.097040 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:58.097220 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:58.097260 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:58.097355 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:58.097408 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:58.097446 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:58.097478 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:58.097515 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:58.097539 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:58.102891 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0208 04:09:58.104023 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:09:58.104249 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:09:58.104287 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:09:58.104333 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:09:58.104382 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:09:58.104464 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:09:58.104502 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:09:58.104541 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:09:58.104566 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:09:58.119017 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:09:58.120880 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:09:58.120951 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0208 04:10:01.259292 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:05.554464 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0208 04:10:05.559684 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:05.561359 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:05.561404 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:05.561393 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:05.561504 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:05.559947 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:05.573761 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:05.578977 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:05.580903 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:05.582555 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:05.583604 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:05.583817 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:05.583913 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:05.584280 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:05.584717 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:05.639720 3399 target.go:112] Terraform output is in /tmp/test1380068977/out --- PASS: TestMinimal (7.54s) === RUN TestMinimal_v1_23 I0208 04:10:05.643850 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:05.644092 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:05.644148 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:05.644200 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:05.644252 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:05.644346 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:05.644378 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:05.644417 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:05.644446 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:05.665209 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0208 04:10:05.667684 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:05.667770 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.174960 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:07.324853 3399 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0208 04:10:07.325590 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:07.325634 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:07.325774 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:07.325892 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:07.326048 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:07.326229 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:07.331024 3399 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0208 04:10:07.336799 3399 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0208 04:10:07.337217 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.337404 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.338486 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.339017 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.339090 3399 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0208 04:10:07.339321 3399 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0208 04:10:07.339445 3399 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0208 04:10:07.379821 3399 target.go:112] Terraform output is in /tmp/test3316609211/out --- PASS: TestMinimal_v1_23 (1.74s) === RUN TestMinimal_v1_24 I0208 04:10:07.382598 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:07.382772 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:07.382809 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:07.382842 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:07.382874 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:07.382933 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:07.382951 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:07.382974 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:07.382991 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:07.410076 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:07.413363 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:07.413458 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.267046 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:08.377773 3399 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0208 04:10:08.379335 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:08.379486 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:08.379776 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:08.379992 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:08.380681 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:08.380842 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:08.382764 3399 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0208 04:10:08.388102 3399 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0208 04:10:08.388349 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.388592 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.389234 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.393072 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.393173 3399 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0208 04:10:08.393456 3399 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0208 04:10:08.393671 3399 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0208 04:10:08.423737 3399 target.go:112] Terraform output is in /tmp/test3290435579/out --- PASS: TestMinimal_v1_24 (1.05s) === RUN TestMinimal_v1_25 I0208 04:10:08.428970 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:08.429238 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:08.429317 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:08.429399 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:08.429478 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:08.429608 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:08.429655 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:08.429714 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:08.429758 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:08.446434 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:08.448404 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:08.448471 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:09.875446 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:10.032548 3399 executor.go:111] Tasks: 0 done / 93 total; 46 can run W0208 04:10:10.033010 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:10.033016 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:10.033107 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:10.033190 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:10.033210 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:10.033287 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:10.034853 3399 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0208 04:10:10.041381 3399 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0208 04:10:10.041698 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:10.041903 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:10.042767 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:10.044566 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:10.044658 3399 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0208 04:10:10.044887 3399 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0208 04:10:10.045022 3399 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0208 04:10:10.087241 3399 target.go:112] Terraform output is in /tmp/test812343367/out --- PASS: TestMinimal_v1_25 (1.66s) === RUN TestMinimal_v1_26 I0208 04:10:10.090189 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:10.090339 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:10.090382 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:10.090416 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:10.090451 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:10.090512 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:10.090529 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:10.090552 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:10.090570 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:10.107346 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:10.110562 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:10.110665 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:11.577017 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:11.713171 3399 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0208 04:10:11.713359 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:11.713595 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:11.713641 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:11.713665 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:11.713804 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:11.713885 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:11.715452 3399 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0208 04:10:11.721979 3399 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0208 04:10:11.722203 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:11.722398 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:11.723414 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:11.723883 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:11.723951 3399 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0208 04:10:11.724197 3399 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0208 04:10:11.724343 3399 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0208 04:10:11.762771 3399 target.go:112] Terraform output is in /tmp/test3743368825/out --- PASS: TestMinimal_v1_26 (1.68s) === RUN TestHetzner I0208 04:10:11.765413 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:12.976343 3399 executor.go:111] Tasks: 0 done / 50 total; 39 can run I0208 04:10:12.979743 3399 executor.go:111] Tasks: 39 done / 50 total; 5 can run I0208 04:10:12.981701 3399 executor.go:111] Tasks: 44 done / 50 total; 2 can run I0208 04:10:12.984792 3399 executor.go:111] Tasks: 46 done / 50 total; 4 can run I0208 04:10:12.986369 3399 executor.go:111] Tasks: 50 done / 50 total; 0 can run I0208 04:10:12.996890 3399 target.go:112] Terraform output is in /tmp/test812797177/out --- PASS: TestHetzner (1.23s) === RUN TestNvidia I0208 04:10:13.000372 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:13.000604 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:13.000652 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:13.000696 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:13.000740 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:13.001142 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:13.001170 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:13.001210 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:13.001232 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:13.026238 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:13.028432 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.028493 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.068457 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:13.147974 3399 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0208 04:10:13.148217 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:13.148248 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:13.148340 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:13.148397 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:13.148484 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:13.148719 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:13.150554 3399 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0208 04:10:13.153590 3399 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0208 04:10:13.153787 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.153986 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.154577 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.156148 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.156257 3399 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0208 04:10:13.156486 3399 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0208 04:10:13.156608 3399 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0208 04:10:13.203189 3399 target.go:112] Terraform output is in /tmp/test1776910091/out --- PASS: TestNvidia (0.21s) === RUN TestMinimalGossip I0208 04:10:13.205700 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:13.205863 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:13.205899 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:13.205933 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:13.205968 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:13.206025 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:13.206043 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:13.206066 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:13.206083 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:13.222008 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.222066 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.319143 3399 executor.go:111] Tasks: 0 done / 86 total; 42 can run I0208 04:10:13.319731 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:13.320761 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:13.321250 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:13.324435 3399 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0208 04:10:13.328555 3399 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0208 04:10:13.328841 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.328935 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.329747 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.331569 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.331674 3399 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0208 04:10:13.331914 3399 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0208 04:10:13.332046 3399 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0208 04:10:13.377834 3399 target.go:112] Terraform output is in /tmp/test846963981/out --- PASS: TestMinimalGossip (0.17s) === RUN TestMinimalGossipIRSA I0208 04:10:13.380318 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:13.380467 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:13.380506 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:13.380538 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:13.380571 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:13.380630 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:13.380651 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:13.380674 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:13.380693 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:13.397265 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.397329 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.506360 3399 executor.go:111] Tasks: 0 done / 89 total; 44 can run W0208 04:10:13.507230 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:13.507274 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:13.507249 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:13.511106 3399 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0208 04:10:13.514266 3399 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0208 04:10:13.514726 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.514951 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.515582 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.516797 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:13.516894 3399 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0208 04:10:13.517124 3399 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0208 04:10:13.517292 3399 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0208 04:10:13.554392 3399 target.go:112] Terraform output is in /tmp/test3561796437/out --- PASS: TestMinimalGossipIRSA (0.18s) === RUN TestMinimalGCE I0208 04:10:13.557854 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:13.559647 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0208 04:10:14.462924 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:14.462990 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:14.463069 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0208 04:10:14.463091 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0208 04:10:14.463105 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:14.463126 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:14.463168 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:14.464267 3399 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0208 04:10:14.469869 3399 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0208 04:10:14.474197 3399 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0208 04:10:14.477005 3399 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0208 04:10:14.477586 3399 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0208 04:10:14.504869 3399 target.go:112] Terraform output is in /tmp/test125514425/out --- PASS: TestMinimalGCE (0.95s) === RUN TestMinimalGCEPrivate I0208 04:10:14.508576 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:14.510232 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:15.309808 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:15.309935 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:15.310051 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0208 04:10:15.310074 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0208 04:10:15.310089 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0208 04:10:15.310111 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0208 04:10:15.310151 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:15.311198 3399 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0208 04:10:15.313305 3399 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0208 04:10:15.315817 3399 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0208 04:10:15.317320 3399 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0208 04:10:15.318019 3399 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0208 04:10:15.318109 3399 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0208 04:10:15.334915 3399 target.go:112] Terraform output is in /tmp/test735744176/out --- PASS: TestMinimalGCEPrivate (0.83s) === RUN TestMinimalGCEInternalLoadBalancer I0208 04:10:15.336859 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:15.337821 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:15.456158 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:15.456206 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:15.456275 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0208 04:10:15.456291 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0208 04:10:15.456302 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0208 04:10:15.456316 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0208 04:10:15.456344 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0208 04:10:15.456365 3399 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0208 04:10:15.457330 3399 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0208 04:10:15.462456 3399 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0208 04:10:15.465439 3399 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0208 04:10:15.465462 3399 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0208 04:10:15.468256 3399 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0208 04:10:15.469334 3399 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0208 04:10:15.469554 3399 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0208 04:10:15.469683 3399 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0208 04:10:15.484373 3399 target.go:112] Terraform output is in /tmp/test599132153/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.15s) === RUN TestMinimalGCELongClusterName I0208 04:10:15.486254 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:15.487169 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0208 04:10:15.672281 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:15.672396 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:15.672529 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0208 04:10:15.672561 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0208 04:10:15.672586 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0208 04:10:15.672618 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0208 04:10:15.672688 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:15.674303 3399 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0208 04:10:15.679346 3399 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0208 04:10:15.682401 3399 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0208 04:10:15.684792 3399 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0208 04:10:15.684949 3399 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0208 04:10:15.707703 3399 target.go:112] Terraform output is in /tmp/test1028931861/out --- PASS: TestMinimalGCELongClusterName (0.22s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0208 04:10:15.710314 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:15.711536 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0208 04:10:15.878220 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:15.878301 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:15.878406 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0208 04:10:15.878428 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0208 04:10:15.878445 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0208 04:10:15.878467 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0208 04:10:15.878512 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0208 04:10:15.878545 3399 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0208 04:10:15.879866 3399 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0208 04:10:15.885194 3399 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0208 04:10:15.887617 3399 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0208 04:10:15.887643 3399 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0208 04:10:15.888929 3399 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0208 04:10:15.889088 3399 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0208 04:10:15.889217 3399 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0208 04:10:15.889325 3399 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0208 04:10:15.906915 3399 target.go:112] Terraform output is in /tmp/test3774779367/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.20s) === RUN TestHA I0208 04:10:15.908927 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:15.909060 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:15.909104 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:15.909138 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:15.909170 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:15.909240 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:15.909253 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:15.909271 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:15.909284 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:15.927538 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:15.929935 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:15.929998 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:15.930035 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:15.930070 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:15.966688 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:16.033383 3399 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0208 04:10:16.033678 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:16.033763 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:16.033877 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:16.034021 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:16.034032 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:16.034050 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:16.034051 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:16.034134 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:16.035588 3399 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0208 04:10:16.041575 3399 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0208 04:10:16.041863 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.041952 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.042026 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.042220 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.042884 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.043080 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.043155 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.044887 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.045004 3399 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0208 04:10:16.045338 3399 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0208 04:10:16.045523 3399 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0208 04:10:16.083274 3399 target.go:112] Terraform output is in /tmp/test926186664/out --- PASS: TestHA (0.18s) === RUN TestHighAvailabilityGCE I0208 04:10:16.088625 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:16.090809 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:16.243799 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:16.243893 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:16.243999 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0208 04:10:16.244032 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0208 04:10:16.244049 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0208 04:10:16.244074 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0208 04:10:16.244131 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0208 04:10:16.244198 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0208 04:10:16.244259 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:16.245926 3399 executor.go:111] Tasks: 0 done / 83 total; 48 can run I0208 04:10:16.247878 3399 executor.go:111] Tasks: 48 done / 83 total; 20 can run I0208 04:10:16.251422 3399 executor.go:111] Tasks: 68 done / 83 total; 7 can run I0208 04:10:16.255224 3399 executor.go:111] Tasks: 75 done / 83 total; 5 can run I0208 04:10:16.256325 3399 executor.go:111] Tasks: 80 done / 83 total; 3 can run I0208 04:10:16.256490 3399 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0208 04:10:16.288321 3399 target.go:112] Terraform output is in /tmp/test582670071/out --- PASS: TestHighAvailabilityGCE (0.20s) === RUN TestComplex I0208 04:10:16.293454 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:16.293676 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:16.293735 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:16.293864 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:16.293920 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:16.293966 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:16.293997 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:16.294038 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:16.294066 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:16.331048 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:16.333534 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:16.333602 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.715185 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:17.853754 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0208 04:10:17.854784 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:17.855186 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:17.855387 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:17.855410 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:17.855500 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:17.857278 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:17.865824 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:17.866146 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.866345 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.867175 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.868890 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.869068 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:17.869389 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:17.869563 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0208 04:10:17.915234 3399 target.go:112] Terraform output is in /tmp/test268394680/out I0208 04:10:17.919451 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:17.919653 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:17.919710 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:17.919763 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:17.919816 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:17.919860 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:17.919890 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:17.919928 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:17.919957 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:17.949527 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:17.952039 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.952120 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:17.990233 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:18.142797 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0208 04:10:18.143115 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.143501 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:18.143534 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:18.143622 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:18.143871 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.145697 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:18.152520 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:18.152809 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.153300 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.154232 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.155741 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.155870 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:18.156211 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:18.156358 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0208 04:10:18.213948 3399 target.go:112] Terraform output is in /tmp/test1189583805/out --- PASS: TestComplex (1.92s) === RUN TestCompress I0208 04:10:18.216661 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:18.216824 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:18.216868 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:18.216902 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:18.216938 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:18.217001 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:18.217019 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:18.217044 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:18.217063 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:18.231594 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:18.233421 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.233481 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.270380 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:18.351029 3399 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0208 04:10:18.351218 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:18.351466 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.352610 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:18.352645 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:18.352770 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:18.355893 3399 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0208 04:10:18.361477 3399 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0208 04:10:18.361757 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.361902 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.363801 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.365579 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.365775 3399 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0208 04:10:18.366283 3399 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0208 04:10:18.366707 3399 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0208 04:10:18.406033 3399 target.go:112] Terraform output is in /tmp/test3002357399/out --- PASS: TestCompress (0.19s) === RUN TestExternalPolicies I0208 04:10:18.408581 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:18.408767 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:18.408815 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:18.408863 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:18.408912 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:18.408996 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:18.409040 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:18.409072 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:18.409096 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:18.437433 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:18.440818 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.440907 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.477606 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:18.568403 3399 executor.go:111] Tasks: 0 done / 100 total; 45 can run W0208 04:10:18.568907 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.568923 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:18.568971 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.568954 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:18.569074 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:18.569121 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:18.570671 3399 executor.go:111] Tasks: 45 done / 100 total; 20 can run I0208 04:10:18.573847 3399 executor.go:111] Tasks: 65 done / 100 total; 30 can run I0208 04:10:18.574141 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.574321 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.574968 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.575498 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.575567 3399 executor.go:111] Tasks: 95 done / 100 total; 3 can run I0208 04:10:18.575849 3399 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0208 04:10:18.575988 3399 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0208 04:10:18.628698 3399 target.go:112] Terraform output is in /tmp/test576039447/out --- PASS: TestExternalPolicies (0.22s) === RUN TestMinimalIPv6 I0208 04:10:18.631287 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:18.631437 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:18.631473 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:18.631505 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:18.631540 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:18.631597 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:18.631613 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:18.631637 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:18.631654 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:18.649495 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:18.651471 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.651532 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.688403 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:18.796034 3399 executor.go:111] Tasks: 0 done / 106 total; 45 can run W0208 04:10:18.796300 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.796533 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:18.796557 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:18.796557 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:18.796639 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:18.796778 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:18.798066 3399 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0208 04:10:18.801142 3399 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0208 04:10:18.801431 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.801634 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.802746 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.802966 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.803050 3399 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0208 04:10:18.803378 3399 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0208 04:10:18.803658 3399 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0208 04:10:18.844362 3399 target.go:112] Terraform output is in /tmp/test1749722367/out --- PASS: TestMinimalIPv6 (0.22s) === RUN TestMinimalIPv6Private I0208 04:10:18.847001 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:18.847147 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:18.847185 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:18.847217 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:18.847249 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:18.847305 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:18.847323 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:18.847344 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:18.847362 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:18.866070 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0208 04:10:18.868250 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.868314 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:18.904611 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:19.012747 3399 executor.go:111] Tasks: 0 done / 124 total; 46 can run W0208 04:10:19.013955 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:19.013954 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:19.014244 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:19.014332 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:19.014470 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:19.015991 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:19.017701 3399 executor.go:111] Tasks: 46 done / 124 total; 23 can run I0208 04:10:19.020729 3399 executor.go:111] Tasks: 69 done / 124 total; 36 can run I0208 04:10:19.021012 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.021255 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.024335 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.025397 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.025528 3399 executor.go:111] Tasks: 105 done / 124 total; 11 can run I0208 04:10:19.025959 3399 executor.go:111] Tasks: 116 done / 124 total; 8 can run I0208 04:10:19.026203 3399 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0208 04:10:19.094672 3399 target.go:112] Terraform output is in /tmp/test1574363016/out --- PASS: TestMinimalIPv6Private (0.25s) === RUN TestMinimalIPv6Calico I0208 04:10:19.099148 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:19.099353 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:19.099407 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:19.099507 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:19.099570 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:19.099615 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:19.099647 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:19.099688 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:19.099721 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:19.129152 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:19.132751 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.132862 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.171759 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:19.562947 3399 executor.go:111] Tasks: 0 done / 109 total; 47 can run I0208 04:10:19.563232 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:19.563283 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:19.563326 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:19.563266 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:19.563369 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:19.563517 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:19.567562 3399 executor.go:111] Tasks: 47 done / 109 total; 21 can run I0208 04:10:19.570491 3399 executor.go:111] Tasks: 68 done / 109 total; 31 can run I0208 04:10:19.570759 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.570936 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.571506 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.572557 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.572644 3399 executor.go:111] Tasks: 99 done / 109 total; 5 can run I0208 04:10:19.572926 3399 executor.go:111] Tasks: 104 done / 109 total; 5 can run I0208 04:10:19.573153 3399 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0208 04:10:19.612696 3399 target.go:112] Terraform output is in /tmp/test1379611801/out --- PASS: TestMinimalIPv6Calico (0.52s) === RUN TestMinimalIPv6Cilium I0208 04:10:19.617352 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:19.617656 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:19.617748 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:19.617833 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:19.617910 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:19.618035 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:19.618091 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:19.618158 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:19.618208 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:19.644544 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:19.647681 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.647766 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.684806 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:19.855976 3399 executor.go:111] Tasks: 0 done / 107 total; 46 can run I0208 04:10:19.856741 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:19.856902 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:19.856787 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:19.856949 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:19.857009 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:19.857095 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:19.858819 3399 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0208 04:10:19.866050 3399 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0208 04:10:19.866293 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.866534 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.867121 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.868957 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.869099 3399 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0208 04:10:19.869488 3399 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0208 04:10:19.869779 3399 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0208 04:10:19.922960 3399 target.go:112] Terraform output is in /tmp/test3537212679/out --- PASS: TestMinimalIPv6Cilium (0.31s) === RUN TestMinimalWarmPool I0208 04:10:19.926134 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:19.926284 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:19.926334 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:19.926368 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:19.926423 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:19.926493 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:19.926510 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:19.926535 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:19.926551 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:19.941702 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:19.943788 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.943847 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:19.980494 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:20.119013 3399 executor.go:111] Tasks: 0 done / 89 total; 45 can run W0208 04:10:20.119412 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.119576 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:20.119725 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:20.119747 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:20.119812 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:20.119841 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.125123 3399 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0208 04:10:20.128953 3399 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0208 04:10:20.129149 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.129262 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.129987 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.131895 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.131987 3399 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0208 04:10:20.132230 3399 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0208 04:10:20.132376 3399 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0208 04:10:20.173177 3399 target.go:112] Terraform output is in /tmp/test3457946082/out --- PASS: TestMinimalWarmPool (0.25s) === RUN TestMinimalEtcd I0208 04:10:20.176009 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:20.176160 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:20.176198 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:20.176231 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:20.176266 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:20.176325 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:20.176343 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:20.176365 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:20.176382 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:20.193506 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:20.195375 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.195433 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.233087 3399 zones.go:93] ListHostedZonesPages { } W0208 04:10:20.323241 3399 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0208 04:10:20.323297 3399 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W0208 04:10:20.324825 3399 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0208 04:10:20.335128 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:20.335435 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:20.335463 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:20.335527 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:20.335605 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:20.335932 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.335941 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:20.337834 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:20.344425 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:20.344651 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.344885 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.345474 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.346964 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.347034 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:20.347233 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:20.347358 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:20.388624 3399 target.go:112] Terraform output is in /tmp/test3764025215/out --- PASS: TestMinimalEtcd (0.22s) === RUN TestMinimalGp3 I0208 04:10:20.393908 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:20.394097 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:20.394138 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:20.394177 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:20.394215 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:20.394280 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:20.394298 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:20.394321 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:20.394340 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:20.409819 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:20.411562 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.411619 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.451096 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:20.519520 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:20.520293 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:20.520333 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:20.520335 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0208 04:10:20.520451 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.520475 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:20.520580 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.523106 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:20.526154 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:20.526550 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.526675 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.527317 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.528909 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.529055 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:20.529358 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:20.529545 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:20.573191 3399 target.go:112] Terraform output is in /tmp/test1833536919/out --- PASS: TestMinimalGp3 (0.18s) === RUN TestMinimalLongClusterName I0208 04:10:20.575691 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:20.575841 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:20.575879 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:20.575913 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:20.575948 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:20.576009 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:20.576029 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:20.576051 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:20.576068 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:20.590235 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:20.591996 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.592052 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0208 04:10:20.629567 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:20.709655 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0208 04:10:20.710180 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.710148 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:20.710344 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:20.710376 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:20.710468 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:20.710697 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.713337 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:20.721967 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:20.722346 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.722518 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.723557 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.723728 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.723811 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:20.724093 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:20.724268 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:20.767962 3399 target.go:112] Terraform output is in /tmp/test4057671075/out --- PASS: TestMinimalLongClusterName (0.20s) === RUN TestExistingSG I0208 04:10:20.774960 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:20.775208 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:20.775271 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:20.775328 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:20.775386 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:20.775488 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:20.775521 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:20.775561 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:20.775590 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:20.810195 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:20.813737 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.813824 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.813882 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.813936 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.851129 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:20.945853 3399 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0208 04:10:20.946181 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:20.946267 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:20.946307 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.946361 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:20.946384 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:20.946450 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:20.946485 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:20.946596 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:20.949450 3399 executor.go:111] Tasks: 53 done / 146 total; 26 can run I0208 04:10:20.958241 3399 executor.go:111] Tasks: 79 done / 146 total; 58 can run I0208 04:10:20.958815 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.959039 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.959321 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.959618 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.960775 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.961223 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.961583 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.962119 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:20.962242 3399 executor.go:111] Tasks: 137 done / 146 total; 5 can run I0208 04:10:20.962682 3399 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0208 04:10:20.963010 3399 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0208 04:10:21.041061 3399 target.go:112] Terraform output is in /tmp/test2050995542/out --- PASS: TestExistingSG (0.27s) === RUN TestBastionAdditionalUserData I0208 04:10:21.046686 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:21.046936 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:21.046998 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:21.047053 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:21.047108 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:21.047205 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:21.047237 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:21.047276 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:21.047306 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:21.081081 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:21.084913 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.085013 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.085080 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.122681 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:21.221182 3399 executor.go:111] Tasks: 0 done / 122 total; 48 can run I0208 04:10:21.221512 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:21.221558 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:21.221687 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:21.221726 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:21.221765 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:21.221883 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:21.221946 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:21.224540 3399 executor.go:111] Tasks: 48 done / 122 total; 29 can run I0208 04:10:21.229817 3399 executor.go:111] Tasks: 77 done / 122 total; 36 can run I0208 04:10:21.230197 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.230291 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.230491 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.230888 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.232409 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.232928 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.233080 3399 executor.go:111] Tasks: 113 done / 122 total; 5 can run I0208 04:10:21.235454 3399 executor.go:111] Tasks: 118 done / 122 total; 4 can run I0208 04:10:21.235797 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0208 04:10:21.298605 3399 target.go:112] Terraform output is in /tmp/test2191805195/out --- PASS: TestBastionAdditionalUserData (0.26s) === RUN TestPrivateWeave I0208 04:10:21.303644 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:21.303886 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:21.303946 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:21.304003 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:21.304060 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:21.304155 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:21.304185 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:21.304225 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:21.304254 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:21.333164 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:21.336448 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.336536 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.336595 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.374275 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:21.495433 3399 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0208 04:10:21.496717 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:21.496832 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:21.496996 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:21.497074 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:21.497114 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:21.497093 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:21.497176 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:21.499092 3399 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0208 04:10:21.505029 3399 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0208 04:10:21.505407 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.506016 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.506313 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.506449 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.507645 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.508940 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.509217 3399 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0208 04:10:21.510001 3399 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0208 04:10:21.510582 3399 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0208 04:10:21.584553 3399 target.go:112] Terraform output is in /tmp/test599004096/out --- PASS: TestPrivateWeave (0.28s) === RUN TestPrivateFlannel I0208 04:10:21.587318 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:21.587464 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:21.587501 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:21.587564 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:21.587601 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:21.587628 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:21.587645 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:21.587667 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:21.587686 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:21.604217 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:21.606212 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.606267 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.606303 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.645924 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:21.820727 3399 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0208 04:10:21.822265 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:21.822797 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:21.823018 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:21.823950 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:21.823985 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:21.824114 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:21.824528 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:21.827676 3399 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0208 04:10:21.833787 3399 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0208 04:10:21.834211 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.834447 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.834639 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.834713 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.835743 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.836022 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.836136 3399 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0208 04:10:21.836511 3399 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0208 04:10:21.836778 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:21.900713 3399 target.go:112] Terraform output is in /tmp/test3780184142/out --- PASS: TestPrivateFlannel (0.32s) === RUN TestPrivateCalico I0208 04:10:21.906041 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:21.906284 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:21.906344 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:21.906451 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:21.906507 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:21.906551 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:21.906584 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:21.906624 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:21.906653 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:21.936665 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:21.940230 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.940313 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.940379 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:21.977887 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:22.357148 3399 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0208 04:10:22.357507 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:22.357529 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:22.357615 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:22.357981 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:22.358045 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:22.358092 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:22.358153 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:22.359868 3399 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0208 04:10:22.365962 3399 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0208 04:10:22.366536 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.366689 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.366879 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.366965 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.367365 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.369573 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.369691 3399 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0208 04:10:22.371014 3399 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0208 04:10:22.371425 3399 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0208 04:10:22.433882 3399 target.go:112] Terraform output is in /tmp/test871844313/out --- PASS: TestPrivateCalico (0.53s) === RUN TestPrivateCilium I0208 04:10:22.437295 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:22.437458 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:22.437496 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:22.437528 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:22.437563 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:22.437621 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:22.437639 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:22.437661 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:22.437679 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:22.454690 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:22.456826 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.456885 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.456924 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.493671 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:22.604489 3399 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0208 04:10:22.604702 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:22.606251 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:22.608121 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:22.608222 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:22.608394 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:22.608430 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:22.609358 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:22.611348 3399 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0208 04:10:22.616036 3399 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0208 04:10:22.616336 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.616608 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.616805 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.617006 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.617241 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.618003 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.618093 3399 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0208 04:10:22.618791 3399 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0208 04:10:22.619189 3399 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0208 04:10:22.680199 3399 target.go:112] Terraform output is in /tmp/test1462324462/out --- PASS: TestPrivateCilium (0.25s) === RUN TestPrivateCilium2 I0208 04:10:22.682971 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:22.683127 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:22.683173 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:22.683243 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:22.683279 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:22.683309 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:22.683326 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:22.683351 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:22.683371 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:22.700596 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:22.702833 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.702893 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.702933 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:22.739827 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:23.351961 3399 executor.go:111] Tasks: 0 done / 124 total; 51 can run W0208 04:10:23.352585 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:23.352797 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:23.354900 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:23.354943 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:23.354990 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:23.355049 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:23.355273 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:23.358338 3399 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0208 04:10:23.361520 3399 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0208 04:10:23.362232 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.362463 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.362725 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.362897 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.363773 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.363848 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.363921 3399 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0208 04:10:23.365181 3399 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0208 04:10:23.365803 3399 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0208 04:10:23.438716 3399 target.go:112] Terraform output is in /tmp/test2590205759/out --- PASS: TestPrivateCilium2 (0.76s) === RUN TestPrivateCiliumAdvanced I0208 04:10:23.443632 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:23.443783 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:23.443830 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:23.443866 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:23.443905 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:23.443970 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:23.443987 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:23.444011 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:23.444030 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:23.464036 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:23.466176 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.466232 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.466271 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.502124 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:23.644945 3399 executor.go:111] Tasks: 0 done / 126 total; 53 can run W0208 04:10:23.645240 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:23.645843 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:23.645912 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:23.645867 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:23.645943 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:23.645952 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:23.645982 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:23.648684 3399 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0208 04:10:23.654475 3399 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0208 04:10:23.655789 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.655930 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.655991 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.656155 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.657154 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.657773 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.657848 3399 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0208 04:10:23.658324 3399 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0208 04:10:23.658590 3399 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0208 04:10:23.722264 3399 target.go:112] Terraform output is in /tmp/test1705838867/out --- PASS: TestPrivateCiliumAdvanced (0.28s) === RUN TestPrivateCanal I0208 04:10:23.727452 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:23.727688 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:23.727743 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:23.727851 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:23.727906 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:23.727948 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:23.727976 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:23.728013 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:23.728040 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:23.754060 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:23.756321 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.756383 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.756421 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:23.793326 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:24.143724 3399 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0208 04:10:24.144153 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:24.144181 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:24.144276 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:24.144459 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:24.144545 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:24.144650 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.144774 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:24.149929 3399 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0208 04:10:24.154190 3399 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0208 04:10:24.154383 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.154663 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.154762 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.154881 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.156108 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.156320 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:24.156440 3399 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0208 04:10:24.156860 3399 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0208 04:10:24.157143 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:24.208255 3399 target.go:112] Terraform output is in /tmp/test3974465653/out --- PASS: TestPrivateCanal (0.49s) === RUN TestPrivateKopeio I0208 04:10:24.216614 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:24.216879 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:24.216930 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:24.216980 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:24.217033 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:24.217120 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:24.217151 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:24.217190 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:24.217217 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:24.248079 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:24.252155 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.252262 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.252363 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.289283 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:24.397881 3399 executor.go:111] Tasks: 0 done / 126 total; 47 can run W0208 04:10:24.398443 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:24.398500 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.398642 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:24.398717 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.398685 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:24.398784 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:24.398825 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:24.400712 3399 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0208 04:10:24.409409 3399 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0208 04:10:24.409805 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.410057 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.410260 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.410358 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.410576 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.413182 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.413304 3399 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0208 04:10:24.413718 3399 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0208 04:10:24.413902 3399 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0208 04:10:24.477394 3399 target.go:112] Terraform output is in /tmp/test1214260599/out --- PASS: TestPrivateKopeio (0.27s) === RUN TestUnmanaged I0208 04:10:24.483784 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:24.484032 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:24.484085 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:24.484145 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:24.484203 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:24.484304 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:24.484340 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:24.484384 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:24.484420 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:24.517416 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:24.520740 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.520841 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.520900 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.560335 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:24.645585 3399 executor.go:111] Tasks: 0 done / 108 total; 45 can run W0208 04:10:24.645899 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:24.646020 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.646561 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:24.647595 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.647756 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:24.647780 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:24.647865 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:24.649514 3399 executor.go:111] Tasks: 45 done / 108 total; 25 can run I0208 04:10:24.653609 3399 executor.go:111] Tasks: 70 done / 108 total; 31 can run I0208 04:10:24.653833 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.653992 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.654204 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.654332 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.655224 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.655318 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.655439 3399 executor.go:111] Tasks: 101 done / 108 total; 4 can run I0208 04:10:24.655752 3399 executor.go:111] Tasks: 105 done / 108 total; 3 can run I0208 04:10:24.655947 3399 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0208 04:10:24.715934 3399 target.go:112] Terraform output is in /tmp/test1563536491/out --- PASS: TestUnmanaged (0.24s) === RUN TestPrivateSharedSubnet I0208 04:10:24.721556 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:24.721744 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:24.721787 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:24.721826 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:24.721861 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:24.721929 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:24.721948 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:24.721973 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:24.721992 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:24.742073 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:24.744417 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.744507 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.744580 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.781838 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:24.870516 3399 executor.go:111] Tasks: 0 done / 107 total; 45 can run W0208 04:10:24.870749 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:24.870971 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.871007 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:24.871043 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:24.871089 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:24.871037 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:24.871125 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:24.873001 3399 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0208 04:10:24.873180 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:24.876563 3399 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0208 04:10:24.876728 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.876904 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.877037 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.877138 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.877598 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.878115 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.878197 3399 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0208 04:10:24.878518 3399 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0208 04:10:24.878692 3399 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0208 04:10:24.919930 3399 target.go:112] Terraform output is in /tmp/test3729484288/out --- PASS: TestPrivateSharedSubnet (0.20s) === RUN TestPrivateSharedIP I0208 04:10:24.923991 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:24.924235 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:24.924280 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:24.924316 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:24.924351 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:24.924414 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:24.924432 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:24.924456 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:24.924475 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:24.953167 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:24.955105 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.955168 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.955205 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:24.997967 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:25.078407 3399 executor.go:111] Tasks: 0 done / 116 total; 45 can run W0208 04:10:25.078650 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.078666 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:25.078690 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0208 04:10:25.080063 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:25.080469 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.078777 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:25.081453 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:25.084535 3399 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0208 04:10:25.084837 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:25.087945 3399 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0208 04:10:25.088247 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.088661 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.089376 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.089535 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.089855 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.090561 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:25.090681 3399 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0208 04:10:25.091452 3399 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0208 04:10:25.091931 3399 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0208 04:10:25.150819 3399 target.go:112] Terraform output is in /tmp/test3620087317/out --- PASS: TestPrivateSharedIP (0.23s) === RUN TestPrivateDns1 I0208 04:10:25.153500 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:25.153652 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:25.153690 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:25.153723 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:25.153757 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:25.153817 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:25.153836 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:25.153860 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:25.153877 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:25.172933 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.172998 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.173034 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.280761 3399 executor.go:111] Tasks: 0 done / 120 total; 46 can run W0208 04:10:25.281087 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:25.281250 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.282805 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:25.283129 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.286080 3399 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0208 04:10:25.286361 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0208 04:10:25.286392 3399 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0208 04:10:25.286450 3399 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0208 04:10:25.286461 3399 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0208 04:10:25.286509 3399 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0208 04:10:25.286777 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.286923 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.288510 3399 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0208 04:10:25.288604 3399 api.go:53] PageSize not implemented W0208 04:10:25.288748 3399 api.go:53] PageSize not implemented I0208 04:10:25.294007 3399 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0208 04:10:25.294185 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.294308 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.295101 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.295566 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.295635 3399 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0208 04:10:25.295862 3399 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0208 04:10:25.295999 3399 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0208 04:10:25.356103 3399 target.go:112] Terraform output is in /tmp/test1642218325/out --- PASS: TestPrivateDns1 (0.21s) === RUN TestPrivateDns2 I0208 04:10:25.358927 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:25.359113 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:25.359159 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:25.359206 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:25.359258 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:25.359331 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:25.359367 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:25.359407 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:25.359433 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:25.379972 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.380060 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.380124 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.492582 3399 executor.go:111] Tasks: 0 done / 116 total; 44 can run W0208 04:10:25.492991 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.493027 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:25.493074 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:25.496226 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.498070 3399 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0208 04:10:25.498277 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:25.498364 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0208 04:10:25.498419 3399 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0208 04:10:25.498492 3399 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0208 04:10:25.498513 3399 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0208 04:10:25.498586 3399 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0208 04:10:25.498952 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.499090 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.500678 3399 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0208 04:10:25.500711 3399 api.go:53] PageSize not implemented W0208 04:10:25.500868 3399 api.go:53] PageSize not implemented I0208 04:10:25.505078 3399 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0208 04:10:25.505273 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.505458 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.506144 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.507741 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.507840 3399 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0208 04:10:25.508089 3399 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0208 04:10:25.508250 3399 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0208 04:10:25.568466 3399 target.go:112] Terraform output is in /tmp/test4075986540/out --- PASS: TestPrivateDns2 (0.21s) === RUN TestDiscoveryFeatureGate I0208 04:10:25.571416 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:25.571585 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:25.571629 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:25.571707 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:25.571748 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:25.571780 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:25.571805 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:25.571829 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:25.571857 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:25.592811 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:25.596022 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.596111 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.632571 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:25.776954 3399 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0208 04:10:25.777505 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:25.777564 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.777644 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:25.777752 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:25.777773 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:25.777860 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:25.781671 3399 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0208 04:10:25.784910 3399 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0208 04:10:25.785104 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.785194 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.785888 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.786713 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.786778 3399 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0208 04:10:25.786947 3399 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0208 04:10:25.787048 3399 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0208 04:10:25.823262 3399 target.go:112] Terraform output is in /tmp/test473755259/out --- PASS: TestDiscoveryFeatureGate (0.26s) === RUN TestVFSServiceAccountIssuerDiscovery I0208 04:10:25.826652 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:25.826816 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:25.826861 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:25.826897 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:25.826935 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:25.827004 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:25.827023 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:25.827048 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:25.827067 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:25.842257 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:25.844029 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.844086 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.886971 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:25.983341 3399 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0208 04:10:25.983786 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:25.983849 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:25.984099 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:25.984107 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:25.984132 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:25.984221 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:25.985651 3399 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0208 04:10:25.992228 3399 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0208 04:10:25.992593 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.992835 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.993617 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.994144 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:25.994210 3399 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0208 04:10:25.994456 3399 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0208 04:10:25.994590 3399 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0208 04:10:26.030332 3399 target.go:112] Terraform output is in /tmp/test4216363258/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.21s) === RUN TestAWSLBController I0208 04:10:26.034664 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:26.034840 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:26.034886 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:26.034958 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:26.034995 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:26.035028 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:26.035046 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:26.035070 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:26.035089 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:26.051007 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:26.052858 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.052916 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.090708 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:26.490772 3399 executor.go:111] Tasks: 0 done / 96 total; 49 can run I0208 04:10:26.491428 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:26.491460 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:26.491455 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:26.491563 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:26.491599 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:26.491563 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:26.495063 3399 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0208 04:10:26.501458 3399 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0208 04:10:26.501799 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.502012 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.503031 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.503361 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.503444 3399 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0208 04:10:26.503674 3399 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0208 04:10:26.503851 3399 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0208 04:10:26.537673 3399 target.go:112] Terraform output is in /tmp/test1960654323/out --- PASS: TestAWSLBController (0.51s) === RUN TestManyAddons I0208 04:10:26.547323 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:26.547566 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:26.547618 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:26.547716 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:26.547765 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:26.547807 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:26.547837 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:26.547874 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:26.547900 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:26.574033 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:26.577313 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.577394 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:26.614306 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:27.164421 3399 executor.go:111] Tasks: 0 done / 95 total; 50 can run I0208 04:10:27.164816 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:27.164844 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:27.164949 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:27.165012 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:27.166540 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:27.166969 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:27.170151 3399 executor.go:111] Tasks: 50 done / 95 total; 19 can run I0208 04:10:27.173075 3399 executor.go:111] Tasks: 69 done / 95 total; 22 can run I0208 04:10:27.173246 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.173436 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.174182 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.174682 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.174749 3399 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0208 04:10:27.174945 3399 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0208 04:10:27.175061 3399 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0208 04:10:27.209505 3399 target.go:112] Terraform output is in /tmp/test154882738/out --- PASS: TestManyAddons (0.67s) === RUN TestManyAddonsCCMIRSA I0208 04:10:27.214510 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:27.214679 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:27.214721 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:27.214787 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:27.214821 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:27.214852 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:27.214869 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:27.214892 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:27.214910 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:27.231502 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:27.233770 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.233831 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.270321 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:27.884514 3399 executor.go:111] Tasks: 0 done / 111 total; 59 can run W0208 04:10:27.885322 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:27.885425 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:27.885396 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:27.885458 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:27.885553 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:27.885436 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:27.889185 3399 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0208 04:10:27.894389 3399 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0208 04:10:27.894790 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.895046 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.895870 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.896771 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:27.896851 3399 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0208 04:10:27.897104 3399 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0208 04:10:27.897247 3399 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0208 04:10:27.950203 3399 target.go:112] Terraform output is in /tmp/test2063248566/out --- PASS: TestManyAddonsCCMIRSA (0.75s) === RUN TestManyAddonsCCMIRSA23 I0208 04:10:27.964911 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:27.965279 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:27.965355 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:27.965511 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:27.965595 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:27.965686 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:27.965755 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:27.965841 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:27.965877 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:28.005733 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0208 04:10:28.008576 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.008637 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.044792 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:28.678560 3399 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0208 04:10:28.679101 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:28.679112 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:28.679127 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:28.679214 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:28.679338 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:28.679361 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:28.681668 3399 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0208 04:10:28.687873 3399 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0208 04:10:28.688110 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.688306 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.689021 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.689794 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.689870 3399 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0208 04:10:28.690099 3399 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0208 04:10:28.690218 3399 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0208 04:10:28.728568 3399 target.go:112] Terraform output is in /tmp/test2359432322/out --- PASS: TestManyAddonsCCMIRSA23 (0.77s) === RUN TestManyAddonsCCMIRSA24 I0208 04:10:28.735475 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:28.735649 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:28.735706 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:28.735779 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:28.735815 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:28.735845 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:28.735864 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:28.735889 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:28.735908 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:28.754873 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:28.757165 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.757244 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:28.793946 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:29.358272 3399 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0208 04:10:29.361917 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:29.362718 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:29.363295 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:29.361901 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:29.363923 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:29.364000 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:29.365707 3399 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0208 04:10:29.371055 3399 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0208 04:10:29.371325 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.371575 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.374517 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.378224 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.378369 3399 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0208 04:10:29.378658 3399 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0208 04:10:29.378821 3399 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0208 04:10:29.423024 3399 target.go:112] Terraform output is in /tmp/test1150696311/out --- PASS: TestManyAddonsCCMIRSA24 (0.70s) === RUN TestManyAddonsCCMIRSA25 I0208 04:10:29.434071 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:29.434291 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:29.434338 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:29.434430 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:29.434476 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:29.434513 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:29.434539 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:29.434571 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:29.434597 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:29.461440 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:29.464716 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.464791 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:29.504046 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:30.098961 3399 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0208 04:10:30.099394 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:30.099605 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:30.099710 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:30.099747 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:30.099730 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:30.099821 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:30.104564 3399 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0208 04:10:30.107646 3399 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0208 04:10:30.107849 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.107987 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.108738 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.109410 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.109526 3399 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0208 04:10:30.109807 3399 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0208 04:10:30.109942 3399 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0208 04:10:30.177303 3399 target.go:112] Terraform output is in /tmp/test2604673051/out --- PASS: TestManyAddonsCCMIRSA25 (0.76s) === RUN TestManyAddonsCCMIRSA26 I0208 04:10:30.189476 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:30.189723 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:30.189787 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:30.189898 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:30.189956 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:30.190006 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:30.190035 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:30.190074 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:30.190104 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:30.217684 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:30.222494 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.222715 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.262510 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:30.936388 3399 executor.go:111] Tasks: 0 done / 111 total; 59 can run W0208 04:10:30.936643 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:30.936686 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:30.936665 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:30.937013 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:30.937046 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:30.937154 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:30.942906 3399 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0208 04:10:30.946886 3399 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0208 04:10:30.947280 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.947458 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.948357 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.949586 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:30.949692 3399 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0208 04:10:30.949913 3399 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0208 04:10:30.950098 3399 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0208 04:10:30.995485 3399 target.go:112] Terraform output is in /tmp/test159031585/out --- PASS: TestManyAddonsCCMIRSA26 (0.81s) === RUN TestCCM I0208 04:10:31.003729 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:31.003917 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:31.003961 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:31.004045 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:31.004081 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:31.004115 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:31.004131 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:31.004156 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:31.004174 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:31.023065 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:31.025314 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.025406 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.062754 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:31.726648 3399 executor.go:111] Tasks: 0 done / 96 total; 51 can run W0208 04:10:31.726937 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:31.726984 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:31.727205 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:31.728815 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:31.728844 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:31.728920 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:31.732933 3399 executor.go:111] Tasks: 51 done / 96 total; 19 can run I0208 04:10:31.736798 3399 executor.go:111] Tasks: 70 done / 96 total; 22 can run I0208 04:10:31.737179 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.737415 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.738315 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.740101 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.740255 3399 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0208 04:10:31.740955 3399 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0208 04:10:31.741436 3399 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0208 04:10:31.773009 3399 target.go:112] Terraform output is in /tmp/test78085106/out --- PASS: TestCCM (0.77s) === RUN TestExternalDNS I0208 04:10:31.778701 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:31.778861 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:31.778906 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:31.778944 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:31.778981 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:31.779056 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:31.779074 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:31.779098 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:31.779118 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:31.804053 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:31.807826 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.807916 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.845661 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:31.913407 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0208 04:10:31.913810 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:31.913906 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:31.913997 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:31.914035 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:31.914023 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:31.914094 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:31.915087 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:31.917980 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:31.918097 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.918337 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.918734 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.919654 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.919723 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:31.919900 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:31.920019 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:31.950840 3399 target.go:112] Terraform output is in /tmp/test3916706794/out --- PASS: TestExternalDNS (0.17s) === RUN TestExternalDNSIRSA I0208 04:10:31.953331 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:31.953478 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:31.953520 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:31.953552 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:31.953587 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:31.953646 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:31.953663 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:31.953686 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:31.953705 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:31.968649 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:31.970444 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:31.970499 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.007639 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:32.080298 3399 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0208 04:10:32.080661 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:32.080685 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:32.080761 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:32.080953 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.080994 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:32.081063 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.082062 3399 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0208 04:10:32.087413 3399 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0208 04:10:32.087593 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.087708 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.088315 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.088983 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.089105 3399 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0208 04:10:32.089441 3399 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0208 04:10:32.089594 3399 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0208 04:10:32.121931 3399 target.go:112] Terraform output is in /tmp/test2891955841/out --- PASS: TestExternalDNSIRSA (0.17s) === RUN TestKarpenter I0208 04:10:32.124710 3399 featureflag.go:160] FeatureFlag "Karpenter"=true I0208 04:10:32.124738 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:32.124879 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:32.124917 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:32.124979 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:32.125013 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:32.125040 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:32.125058 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:32.125080 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:32.125098 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:32.143479 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:32.145516 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.145566 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.145602 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.181753 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:32.296124 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.296216 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.296284 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.398526 3399 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0208 04:10:32.399163 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:32.399192 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0208 04:10:32.399250 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.399258 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:32.399296 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:32.399416 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.401626 3399 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0208 04:10:32.406880 3399 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0208 04:10:32.407109 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.407351 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.407460 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.408180 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.409596 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.412695 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.412812 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.413364 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.413468 3399 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0208 04:10:32.413842 3399 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0208 04:10:32.414023 3399 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0208 04:10:32.468778 3399 target.go:112] Terraform output is in /tmp/test3958918962/out I0208 04:10:32.475546 3399 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.35s) === RUN TestSharedSubnet I0208 04:10:32.475716 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:32.475967 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:32.476031 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:32.476088 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:32.476147 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:32.476243 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:32.476272 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:32.476314 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:32.476344 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:32.502276 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:32.505430 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.505521 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.543354 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:32.632667 3399 executor.go:111] Tasks: 0 done / 80 total; 42 can run W0208 04:10:32.633410 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.633417 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:32.633542 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.633373 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:32.633460 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:32.633661 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:32.636495 3399 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0208 04:10:32.636824 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:32.643130 3399 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0208 04:10:32.643430 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.643654 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.644114 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.644921 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.644990 3399 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0208 04:10:32.645169 3399 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0208 04:10:32.645282 3399 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0208 04:10:32.671557 3399 target.go:112] Terraform output is in /tmp/test1654758940/out --- PASS: TestSharedSubnet (0.20s) === RUN TestSharedVPC I0208 04:10:32.674047 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:32.674197 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:32.674234 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:32.674266 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:32.674302 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:32.674361 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:32.674378 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:32.674402 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:32.674419 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:32.691258 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:32.692960 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.693018 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.729327 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:32.796650 3399 executor.go:111] Tasks: 0 done / 84 total; 42 can run W0208 04:10:32.797074 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:32.797225 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:32.797288 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:32.797779 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:32.797822 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:32.797983 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:32.799603 3399 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0208 04:10:32.800014 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:32.804852 3399 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0208 04:10:32.805355 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.805570 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.806726 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.807578 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:32.807693 3399 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0208 04:10:32.807998 3399 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0208 04:10:32.808198 3399 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0208 04:10:32.846743 3399 target.go:112] Terraform output is in /tmp/test1001228890/out --- PASS: TestSharedVPC (0.18s) === RUN TestExistingIAM I0208 04:10:32.851036 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:32.851277 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:32.851334 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:32.851386 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:32.851441 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:32.851534 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:32.851565 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:32.851604 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:32.851637 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:32.885759 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:32.889102 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:32.889194 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:32.889258 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:32.889314 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:32.926336 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:33.049255 3399 executor.go:111] Tasks: 0 done / 101 total; 51 can run I0208 04:10:33.049560 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:33.049733 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:33.049767 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:33.049863 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:33.049978 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.050175 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.050286 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.051551 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.052342 3399 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0208 04:10:33.064033 3399 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0208 04:10:33.064306 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.064782 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.065088 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.065724 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.066035 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.066969 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.067306 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.068354 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:33.068466 3399 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0208 04:10:33.072277 3399 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0208 04:10:33.072819 3399 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0208 04:10:33.140664 3399 target.go:112] Terraform output is in /tmp/test1536988648/out --- PASS: TestExistingIAM (0.29s) === RUN TestPhaseNetwork I0208 04:10:33.145454 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.145733 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.145787 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.145837 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.145887 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.145932 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.145964 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.145999 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.146020 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:33.163382 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:33.165323 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.165381 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.165418 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.203845 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:33.290949 3399 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0208 04:10:33.291241 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:33.291307 3399 executor.go:111] Tasks: 46 done / 119 total; 28 can run I0208 04:10:33.291609 3399 executor.go:111] Tasks: 74 done / 119 total; 36 can run I0208 04:10:33.291925 3399 executor.go:111] Tasks: 110 done / 119 total; 5 can run I0208 04:10:33.292093 3399 executor.go:111] Tasks: 115 done / 119 total; 4 can run I0208 04:10:33.292221 3399 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0208 04:10:33.299775 3399 target.go:112] Terraform output is in /tmp/test3490131850/out --- PASS: TestPhaseNetwork (0.15s) === RUN TestExternalLoadBalancer I0208 04:10:33.300481 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.300681 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.300730 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.300780 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.300830 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.300919 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.300947 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.300978 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.301000 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:33.319474 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:33.322143 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.322209 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.360805 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:33.437575 3399 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0208 04:10:33.438140 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.438226 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:33.438298 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.438271 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:33.438260 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:33.439219 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:33.443567 3399 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0208 04:10:33.448429 3399 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0208 04:10:33.448777 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.448979 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.450079 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.451275 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.451522 3399 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0208 04:10:33.451866 3399 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0208 04:10:33.452028 3399 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0208 04:10:33.492002 3399 target.go:112] Terraform output is in /tmp/test3199954772/out --- PASS: TestExternalLoadBalancer (0.19s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0208 04:10:33.494806 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.494987 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.495028 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.495067 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.495105 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.495166 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.495186 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.495212 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.495231 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:33.517651 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:33.521839 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.521969 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.522070 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.522165 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.522250 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.522347 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.522437 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.559360 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:33.642461 3399 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0208 04:10:33.642803 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.642972 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:33.643001 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0208 04:10:33.643077 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.643089 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:33.643183 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.643248 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.643331 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:33.644940 3399 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0208 04:10:33.651718 3399 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0208 04:10:33.652082 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.652286 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.652506 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.652728 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.653214 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.654524 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.654726 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.654816 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.654905 3399 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0208 04:10:33.655279 3399 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0208 04:10:33.655512 3399 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0208 04:10:33.727942 3399 target.go:112] Terraform output is in /tmp/test636228036/out --- PASS: TestMixedInstancesASG (0.24s) === RUN TestMixedInstancesSpotASG I0208 04:10:33.733579 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.733834 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.733915 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.733997 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.734085 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.734218 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.734268 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.734309 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.734335 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:33.769591 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:33.773008 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773092 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773149 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773207 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773263 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773313 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.773363 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.811464 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:33.916305 3399 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0208 04:10:33.916717 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.916801 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.916900 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:33.916976 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:33.917021 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:33.917120 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:33.917144 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:33.917224 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:33.919159 3399 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0208 04:10:33.922311 3399 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0208 04:10:33.922604 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.923148 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.923388 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.923495 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.926660 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.927529 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.927937 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.928180 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:33.928270 3399 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0208 04:10:33.928610 3399 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0208 04:10:33.928782 3399 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0208 04:10:33.988820 3399 target.go:112] Terraform output is in /tmp/test215157851/out --- PASS: TestMixedInstancesSpotASG (0.26s) === RUN TestAdditionalObjects I0208 04:10:33.994071 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.994308 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.994359 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.994412 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.994470 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.994563 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.994595 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.994635 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.994662 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:34.022688 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0208 04:10:34.024783 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.024908 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.062798 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:34.204812 3399 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0208 04:10:34.205268 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:34.207310 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:34.207611 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.207945 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:34.208001 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:34.208293 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:34.210746 3399 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0208 04:10:34.215748 3399 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0208 04:10:34.215909 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.216124 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.217377 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.217521 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0208 04:10:34.217589 3399 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0208 04:10:34.217863 3399 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0208 04:10:34.218020 3399 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0208 04:10:34.261831 3399 target.go:112] Terraform output is in /tmp/test466651151/out --- PASS: TestAdditionalObjects (0.27s) === RUN TestContainerd I0208 04:10:34.265730 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:34.265919 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:34.265957 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:34.265990 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:34.266023 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:34.266081 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:34.266098 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:34.266120 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:34.266137 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:34.281357 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:34.283142 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.283200 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.319808 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:34.409157 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:34.409547 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:34.409618 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.409665 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:34.409757 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.409702 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:34.409795 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:34.411246 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:34.417187 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:34.417672 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.417898 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.418780 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.419973 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.420072 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:34.420311 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:34.420462 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:34.459292 3399 target.go:112] Terraform output is in /tmp/test164252053/out --- PASS: TestContainerd (0.20s) === RUN TestContainerdCustom I0208 04:10:34.461976 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:34.462131 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:34.462174 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:34.462208 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:34.462244 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:34.462316 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:34.462346 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:34.462373 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:34.462391 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:34.480687 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:34.482613 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.482677 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.520215 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:34.598893 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:34.599155 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:34.599233 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.599190 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:34.599388 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.599371 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:34.599474 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:34.601062 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:34.604127 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:34.604514 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.604722 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.605791 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.607050 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.607152 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:34.607415 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:34.607632 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:34.666027 3399 target.go:112] Terraform output is in /tmp/test3305947940/out --- PASS: TestContainerdCustom (0.21s) === RUN TestDockerCustom I0208 04:10:34.670349 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:34.670593 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:34.670639 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:34.670685 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:34.670733 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:34.670822 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:34.670858 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:34.670893 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:34.670922 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:34.696294 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:34.699311 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.699397 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.737526 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:34.834376 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:34.835023 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0208 04:10:34.835066 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.835061 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:34.835152 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:34.835188 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:34.835395 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:34.837118 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:34.840015 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:34.840311 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.840505 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.841784 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.842973 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:34.843141 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:34.843387 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:34.843516 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:34.893917 3399 target.go:112] Terraform output is in /tmp/test900013794/out --- PASS: TestDockerCustom (0.23s) === RUN TestAPIServerNodes I0208 04:10:34.903259 3399 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0208 04:10:34.903298 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:34.903548 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:34.903602 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:34.903707 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:34.903762 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:34.903810 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:34.903841 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:34.903881 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:34.903910 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:34.932318 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0208 04:10:34.935498 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:34.935591 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:34.935658 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.059462 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:36.177928 3399 executor.go:111] Tasks: 0 done / 100 total; 47 can run W0208 04:10:36.178255 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:36.178373 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:36.178408 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:36.178448 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:36.178521 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:36.178542 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:36.178607 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:36.180340 3399 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0208 04:10:36.183330 3399 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0208 04:10:36.183643 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.183784 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.183945 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.185604 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.187102 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.187189 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.187755 3399 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0208 04:10:36.187979 3399 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0208 04:10:36.188139 3399 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0208 04:10:36.233229 3399 target.go:112] Terraform output is in /tmp/test2363881808/out I0208 04:10:36.238624 3399 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.34s) === RUN TestNTHQueueProcessor I0208 04:10:36.238742 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:36.238914 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:36.238957 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:36.238994 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:36.239052 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:36.239113 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:36.239130 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:36.239154 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:36.239172 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:36.254809 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0208 04:10:36.256645 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:36.256706 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.547487 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:37.639407 3399 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0208 04:10:37.639747 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:37.640149 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:37.640163 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:37.640284 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:37.640310 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:37.640415 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:37.641699 3399 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0208 04:10:37.646559 3399 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0208 04:10:37.647459 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.647622 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.648179 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.648978 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.649061 3399 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0208 04:10:37.649270 3399 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0208 04:10:37.649480 3399 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0208 04:10:37.696838 3399 target.go:112] Terraform output is in /tmp/test1949391465/out --- PASS: TestNTHQueueProcessor (1.46s) === RUN TestCustomIRSA I0208 04:10:37.699933 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:37.700094 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:37.700137 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:37.700170 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:37.700205 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:37.700266 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:37.700284 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:37.700310 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:37.700329 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:37.717944 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0208 04:10:37.719995 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.720057 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:37.758139 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.137947 3399 executor.go:111] Tasks: 0 done / 100 total; 50 can run I0208 04:10:38.138398 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:38.138421 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:38.138490 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0208 04:10:38.138524 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:38.138603 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:38.138664 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:38.143604 3399 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0208 04:10:38.146363 3399 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0208 04:10:38.146612 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.146758 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.147329 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.147918 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.148015 3399 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0208 04:10:38.148319 3399 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0208 04:10:38.148520 3399 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0208 04:10:38.186084 3399 target.go:112] Terraform output is in /tmp/test2943232852/out --- PASS: TestCustomIRSA (0.49s) === RUN TestClusterNameDigit I0208 04:10:38.191189 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:38.191349 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:38.191385 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:38.191417 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:38.191449 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:38.191504 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:38.191521 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:38.191543 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:38.191558 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:38.207390 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0208 04:10:38.209230 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.209284 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.247983 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.328816 3399 executor.go:111] Tasks: 0 done / 95 total; 47 can run I0208 04:10:38.329037 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0208 04:10:38.329354 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target W0208 04:10:38.329354 3399 warmpool.go:124] ASG warm pool is not supported by the terraform target I0208 04:10:38.329618 3399 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0208 04:10:38.329638 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:38.329705 3399 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0208 04:10:38.332941 3399 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0208 04:10:38.336388 3399 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0208 04:10:38.336614 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.336796 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.337959 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.338520 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.338598 3399 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0208 04:10:38.338831 3399 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0208 04:10:38.338944 3399 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0208 04:10:38.386599 3399 target.go:112] Terraform output is in /tmp/test2640407999/out --- PASS: TestClusterNameDigit (0.20s) === RUN TestLifecycleMinimalAWS I0208 04:10:38.391145 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:38.391392 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:38.391444 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:38.391499 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:38.391550 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:38.391588 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:38.391622 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:38.391657 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:38.391683 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:38.391742 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0208 04:10:38.411396 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:38.413103 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.413159 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0208 04:10:38.449699 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.521636 3399 iamrole.go:89] ListRoles: { } I0208 04:10:38.523150 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:38.523363 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:38.523329 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.523552 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.523742 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0208 04:10:38.523819 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.523844 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0208 04:10:38.523867 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.524021 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0208 04:10:38.524101 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0208 04:10:38.524083 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.524193 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:38.524232 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.524335 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:38.524424 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:38.524458 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:38.524502 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:38.524591 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:38.524646 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:38.524668 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:38.524699 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.524728 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.524912 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:38.525050 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:38.525462 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:38.525590 3399 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0208 04:10:38.525567 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.525628 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0208 04:10:38.525682 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.525775 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0208 04:10:38.525774 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0208 04:10:38.525911 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0208 04:10:38.525985 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.526137 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0208 04:10:38.526147 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0208 04:10:38.526138 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0208 04:10:38.526354 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.526536 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.526617 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:38.526662 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.526762 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:38.526835 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:38.526895 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:38.526898 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0208 04:10:38.526937 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:38.527028 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:38.527073 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:38.527126 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:38.527174 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:38.527208 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:38.527287 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:38.527083 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0208 04:10:38.527343 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:38.527403 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0208 04:10:38.527501 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:38.528556 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:38.528693 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.528770 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.528881 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:38.528910 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:38.528948 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.529057 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.529232 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.529343 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.529398 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" }] }] } I0208 04:10:38.529518 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0208 04:10:38.529606 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.529696 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:38.529732 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.529860 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.529938 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:38.530028 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.530116 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0208 04:10:38.530190 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:38.530219 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.530292 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.530897 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.531114 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.531340 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.531583 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0208 04:10:38.531820 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" }] }] } I0208 04:10:38.532046 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:38.532692 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:38.532796 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:38.532824 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.532848 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:38.532888 3399 dns.go:238] Pre-creating DNS records I0208 04:10:38.532907 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:38.536696 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:38.538359 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.538415 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0208 04:10:38.576035 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.641239 3399 iamrole.go:89] ListRoles: { } I0208 04:10:38.642348 3399 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0208 04:10:38.642639 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:38.642652 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.642802 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.642957 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.643839 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:38.643876 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:38.643936 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.643980 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.644093 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:38.644152 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:38.647450 3399 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0208 04:10:38.647554 3399 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0208 04:10:38.647565 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.647634 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0208 04:10:38.647645 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.647729 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:38.647778 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0208 04:10:38.647881 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:38.647915 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.647987 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0208 04:10:38.648066 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:38.648102 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] }] } I0208 04:10:38.648185 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:38.651986 3399 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0208 04:10:38.652105 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.652190 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.652359 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.652528 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.652825 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.652958 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.653131 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:38.654523 3399 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0208 04:10:38.654675 3399 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0208 04:10:38.654747 3399 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0208 04:10:38.657443 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:38.657494 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:38.657528 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:38.657559 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:38.657589 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:38.657623 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:38.657654 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:38.657684 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:38.657714 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:38.657747 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:38.657780 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:38.657814 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:38.657851 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:38.657881 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:38.657911 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:38.657947 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:38.657978 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:38.658011 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:38.658043 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:38.658077 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:38.658110 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:38.658141 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:38.658169 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:38.658201 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:38.658233 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:38.658266 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:38.659419 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:38.659442 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:38.659477 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659521 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0208 04:10:38.659569 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659620 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659663 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659711 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659751 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659789 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0208 04:10:38.659832 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659879 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0208 04:10:38.659915 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0208 04:10:38.659955 3399 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0208 04:10:38.659983 3399 api.go:53] PageSize not implemented I0208 04:10:38.660001 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:38.660028 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:38.660043 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:38.660067 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:38.660085 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:38.660133 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:38.660159 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:38.660201 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:38.660238 3399 iamrole.go:89] ListRoles: { } I0208 04:10:38.660284 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:38.660323 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:38.660353 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:38.660826 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok autoscaling-group:nodes.minimal.example.com ok autoscaling-config:lt-1 ok I0208 04:10:38.660911 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:38.660892 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0208 04:10:38.661000 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:38.661018 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0208 04:10:38.661065 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0208 04:10:38.660984 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0208 04:10:38.661112 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0208 04:10:38.661043 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0208 04:10:38.661199 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:38.661215 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:38.661239 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:38.661275 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0208 04:10:38.661304 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:38.661331 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0208 04:10:38.661395 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:38.661402 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:38.661425 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:38.661488 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0208 04:10:38.661527 3399 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0208 04:10:38.661508 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0208 04:10:38.661539 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } subnet:subnet-1 ok I0208 04:10:38.661560 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0208 04:10:38.661581 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0208 04:10:38.661604 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0208 04:10:38.661611 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0208 04:10:38.661626 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0208 04:10:38.661656 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:38.661701 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:38.661743 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:38.661967 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.27s) === RUN TestLifecycleMinimalOpenstack I0208 04:10:38.662181 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0208 04:10:38.668193 3399 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:38.808599 3399 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0208 04:10:38.808661 3399 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0208 04:10:38.819083 3399 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0208 04:10:38.825494 3399 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0208 04:10:38.886966 3399 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0208 04:10:38.889688 3399 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0208 04:10:38.891704 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:38.891821 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:38.891852 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0208 04:10:38.896858 3399 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:39.049625 3399 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0208 04:10:39.049673 3399 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0208 04:10:39.065549 3399 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0208 04:10:39.071511 3399 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0208 04:10:39.075727 3399 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0208 04:10:39.077031 3399 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0208 04:10:39.079304 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:39.079597 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:39.080779 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0208 04:10:39.082714 3399 executor.go:111] Tasks: 96 done / 96 total; 0 can run Instance:784e357a-f825-47a7-b4b8-2650603cb2fe ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:ba5b27ee-e6eb-491d-b12d-852e0b29384a ok Network:7dd8a400-188e-41ce-b7f6-1c86d9139132 ok ServerGroup:fb634ef1-e78a-48ac-9308-8461ff0d41af ok Subnet:7155e055-7055-432d-9bee-9cbd5b353223 ok Instance:e2cc3ac9-17c8-4cfe-b8e8-dff458c29118 ok SecurityGroup:779664d8-f004-4eb8-ae7f-819eb284c01a ok ServerGroup:35a1b805-8ecb-4f6d-b4f0-823866e5eaee ok Port:f0d2ba49-9bba-4408-b254-a7c2baf92569 ok SecurityGroup:fd4e2100-3b7d-406b-a37e-d8dd30b272d6 ok Port:b6c9216f-f10b-41c9-a62b-d4a901af5204 ok SecurityGroup:040a42d5-6369-4ce7-b807-67378c4991fc ok Instance:7eeadf04-8857-4b44-9970-4c9d58c42572 ok --- PASS: TestLifecycleMinimalOpenstack (1.53s) === RUN TestLifecycleMinimalGCE I0208 04:10:40.194378 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0208 04:10:40.195633 3399 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0208 04:10:40.357265 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:40.357327 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:40.357399 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0208 04:10:40.357418 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0208 04:10:40.357432 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:40.357453 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:40.357500 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:40.358548 3399 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0208 04:10:40.360361 3399 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-nodes" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0208 04:10:40.360396 3399 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0208 04:10:40.363447 3399 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0208 04:10:40.365051 3399 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0208 04:10:40.365203 3399 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0208 04:10:40.365278 3399 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0208 04:10:40.531434 3399 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0208 04:10:40.531510 3399 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0208 04:10:40.531571 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0208 04:10:40.531593 3399 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0208 04:10:40.531609 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:40.531628 3399 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0208 04:10:40.531674 3399 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0208 04:10:40.535014 3399 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0208 04:10:40.537316 3399 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0208 04:10:40.541833 3399 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0208 04:10:40.543528 3399 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0208 04:10:40.543653 3399 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0208 04:10:40.545351 3399 gce.go:98] Scanning zones: [us-test1-a] I0208 04:10:40.545744 3399 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0208 04:10:40.545767 3399 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1675829440 ok InstanceTemplate:nodes-minimal-gce-example-com-1675829440 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.35s) === RUN TestLifecycleFloatingIPOpenstack I0208 04:10:40.547015 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0208 04:10:40.552467 3399 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:40.716040 3399 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0208 04:10:40.716081 3399 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0208 04:10:40.729424 3399 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0208 04:10:40.732313 3399 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0208 04:10:40.781808 3399 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0208 04:10:40.783541 3399 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0208 04:10:40.784945 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:40.785214 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:40.785222 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0208 04:10:40.788149 3399 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0208 04:10:40.954273 3399 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0208 04:10:40.954315 3399 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0208 04:10:40.972812 3399 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0208 04:10:40.975797 3399 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0208 04:10:40.987267 3399 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0208 04:10:40.988633 3399 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0208 04:10:40.990771 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:40.990906 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0208 04:10:40.991066 3399 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0208 04:10:40.992118 3399 executor.go:111] Tasks: 98 done / 98 total; 0 can run Router-IF:905b16b6-cdef-49f0-beac-8530b9723a9d ok SecurityGroup:ac921774-5de6-4c61-a211-3a3be6af27fc ok SecurityGroup:4277a84a-fd6b-4c8d-a818-06b9106a8d3b ok Port:44ddeab6-9528-416e-a146-9269edbf133a ok ServerGroup:ced9cb3a-fb27-4657-8ff7-9aefb7d26271 ok ServerGroup:1863ba6c-dd15-4f5f-b16a-759c543c1e65 ok Port:c9ecd0b4-fa39-4c6b-9fa9-1c0ec75de724 ok Router:891b803e-530f-40d0-9a12-8adef935d4a2 ok Subnet:905b16b6-cdef-49f0-beac-8530b9723a9d ok Port:4894e229-bdd0-4a0d-a948-8e76d23970e7 ok Instance:21979cc0-a9b8-45b7-a306-48360ee9f9d3 ok SecurityGroup:336715f8-fbc3-416a-b025-6ee7f461fc64 ok Port:de137069-cedd-4276-ab1c-3a6214dea774 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:61729e6c-aeae-4c95-b058-77472c240694 ok Network:ce1d40c0-dc35-4c54-9141-67a898eeb21c ok Instance:1fe45418-f93c-4430-b6be-77dfdbd98c4b ok --- PASS: TestLifecycleFloatingIPOpenstack (1.55s) === RUN TestLifecyclePrivateCalico I0208 04:10:42.098259 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:42.098450 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:42.098489 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:42.098525 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:42.098560 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:42.098589 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:42.098605 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:42.098629 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:42.098647 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:42.098700 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0208 04:10:42.130830 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:42.135095 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.135219 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.135316 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.172506 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:42.531967 3399 iamrole.go:89] ListRoles: { } I0208 04:10:42.533440 3399 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0208 04:10:42.533909 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:42.533923 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0208 04:10:42.533950 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.534029 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] } I0208 04:10:42.534062 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.534184 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0208 04:10:42.534247 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0208 04:10:42.534255 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.534307 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0208 04:10:42.534275 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.534357 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0208 04:10:42.534387 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:42.534436 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0208 04:10:42.535621 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:42.537547 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0208 04:10:42.537643 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:42.538039 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:42.538090 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:42.538171 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:42.538199 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:42.538314 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.538427 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:42.538453 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:42.538475 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:42.538503 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.538968 3399 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0208 04:10:42.539099 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.539196 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.539160 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0208 04:10:42.539313 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0208 04:10:42.539282 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:42.539420 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:42.539386 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0208 04:10:42.539468 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:42.539438 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0208 04:10:42.539474 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0208 04:10:42.539512 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0208 04:10:42.539664 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0208 04:10:42.539692 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:42.539751 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:42.539780 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:42.539731 3399 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0208 04:10:42.539805 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.539876 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0208 04:10:42.539912 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:42.539973 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.539989 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0208 04:10:42.540110 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:42.540158 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0208 04:10:42.540343 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0208 04:10:42.540477 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:42.540537 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0208 04:10:42.540604 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.540600 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0208 04:10:42.540660 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0208 04:10:42.540723 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.540767 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.540838 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:42.540869 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:42.540941 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:42.540977 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:42.541004 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:42.541046 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" } ] } I0208 04:10:42.541095 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:42.541133 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:42.541157 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:42.541236 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0208 04:10:42.541268 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:42.541370 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:42.541420 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:42.541457 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:42.541545 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:42.541591 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:42.541629 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:42.541670 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:42.541716 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.541799 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:42.541848 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:42.541887 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] } I0208 04:10:42.541635 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0208 04:10:42.542134 3399 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0208 04:10:42.542256 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0208 04:10:42.542313 3399 api.go:53] PageSize not implemented I0208 04:10:42.542353 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.542429 3399 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.542489 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0208 04:10:42.542585 3399 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" }] } W0208 04:10:42.542643 3399 api.go:53] PageSize not implemented I0208 04:10:42.542524 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.542693 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0208 04:10:42.542677 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0208 04:10:42.542748 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.542842 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0208 04:10:42.542918 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0208 04:10:42.542988 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0208 04:10:42.542993 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.543078 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0208 04:10:42.543106 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.543111 3399 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I0208 04:10:42.543170 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0208 04:10:42.543199 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0208 04:10:42.543244 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0208 04:10:42.543220 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:42.543323 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.543472 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0208 04:10:42.543513 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0208 04:10:42.543556 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.543709 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.543842 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.543971 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.544278 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0208 04:10:42.544453 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.544613 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0208 04:10:42.544761 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.544842 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.544997 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.545163 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.545302 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" }] }] } I0208 04:10:42.545561 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.545732 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.546002 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0208 04:10:42.546184 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.546344 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0208 04:10:42.546469 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0208 04:10:42.546582 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0208 04:10:42.546689 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.546829 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.546967 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.547108 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:42.547227 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:42.547367 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0208 04:10:42.547513 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:42.547564 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.547720 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.547808 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.547965 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0208 04:10:42.548019 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0208 04:10:42.548170 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.548241 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:42.548322 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.550112 3399 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0208 04:10:42.550213 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.550299 3399 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:42.550286 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:42.550453 3399 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0208 04:10:42.550555 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:42.550506 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:42.550606 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:42.550654 3399 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0208 04:10:42.550720 3399 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0208 04:10:42.550834 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.550950 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0208 04:10:42.551003 3399 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0208 04:10:42.551033 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:42.551053 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:42.551083 3399 dns.go:238] Pre-creating DNS records I0208 04:10:42.551097 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:42.556526 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0208 04:10:42.559685 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.559774 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.559828 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.596785 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:42.957711 3399 iamrole.go:89] ListRoles: { } I0208 04:10:42.959200 3399 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0208 04:10:42.959530 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.959644 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.959739 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:42.959820 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:42.959859 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:42.959913 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.959959 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] }] } I0208 04:10:42.960100 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.960225 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:42.960288 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:42.961336 3399 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0208 04:10:42.961492 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0208 04:10:42.961456 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0208 04:10:42.961566 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0208 04:10:42.961588 3399 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0208 04:10:42.961607 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.961732 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:42.961762 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:42.961810 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.961876 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:42.961934 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0208 04:10:42.962038 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:42.962072 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0208 04:10:42.962156 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:42.962184 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] }] } I0208 04:10:42.962260 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:42.962282 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0208 04:10:42.962394 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:42.962427 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0208 04:10:42.962509 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:42.962543 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.962619 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.962693 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.964892 3399 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0208 04:10:42.965076 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.965232 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } W0208 04:10:42.965893 3399 api.go:53] PageSize not implemented I0208 04:10:42.965894 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.965920 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0208 04:10:42.966009 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0208 04:10:42.966005 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.966049 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } W0208 04:10:42.966124 3399 api.go:53] PageSize not implemented I0208 04:10:42.966129 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.966142 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0208 04:10:42.966208 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0208 04:10:42.966205 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:42.966240 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0208 04:10:42.966246 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0208 04:10:42.966280 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0208 04:10:42.966500 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.966600 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.966778 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.966880 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:42.967099 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:42.967235 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.967585 3399 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0208 04:10:42.967644 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0208 04:10:42.967684 3399 api.go:53] PageSize not implemented I0208 04:10:42.967662 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:42.967726 3399 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0208 04:10:42.967694 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:42.967706 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } W0208 04:10:42.967851 3399 api.go:53] PageSize not implemented I0208 04:10:42.967875 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0208 04:10:42.967965 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0208 04:10:42.968035 3399 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0208 04:10:42.968092 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:42.968168 3399 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0208 04:10:42.970459 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:42.970503 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:42.970527 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:42.970548 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:42.970566 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:42.970589 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0208 04:10:42.970611 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:42.970631 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:42.970650 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:42.970670 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:42.970691 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:42.970713 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:42.970737 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:42.970761 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0208 04:10:42.970787 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:42.970808 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:42.970828 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:42.970848 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:42.970870 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:42.970889 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:42.970911 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:42.970933 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:42.970956 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:42.970981 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:42.971002 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:42.971024 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:42.971049 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:42.971070 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:42.971092 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:42.971116 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:42.971136 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:42.971160 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:42.971183 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:42.971205 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:42.972150 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:42.972170 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:42.972197 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972242 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0208 04:10:42.972292 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972345 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972382 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972414 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972443 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972472 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0208 04:10:42.972512 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972550 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:42.972570 3399 natgateway.go:117] DescribeNatGateways: { } I0208 04:10:42.972596 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0208 04:10:42.972624 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0208 04:10:42.972658 3399 api.go:53] PageSize not implemented I0208 04:10:42.972671 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm","api-privatecalico-example-0uch4k"] } I0208 04:10:42.972705 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:42.972722 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:42.972737 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:42.972753 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:42.972765 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:42.972800 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:42.972819 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:42.972848 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:42.972879 3399 iamrole.go:89] ListRoles: { } I0208 04:10:42.972912 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:42.972943 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:42.972970 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0208 04:10:42.972994 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:42.973017 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0208 04:10:42.973418 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok autoscaling-group:nodes.privatecalico.example.com ok I0208 04:10:42.973452 3399 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0208 04:10:42.973409 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0208 04:10:42.973510 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } autoscaling-group:bastion.privatecalico.example.com ok I0208 04:10:42.973551 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0208 04:10:42.973480 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok I0208 04:10:42.973537 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:42.973588 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } load-balancer:bastion-privatecalico-exa-hocohm ok I0208 04:10:42.973654 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0208 04:10:42.973619 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:42.973690 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0208 04:10:42.973735 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0208 04:10:42.973700 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0208 04:10:42.973796 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:42.973807 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok autoscaling-config:lt-1 ok I0208 04:10:42.973866 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:42.973900 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:42.973934 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0208 04:10:42.974060 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:42.974073 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:42.974093 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:42.974128 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0208 04:10:42.974147 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:42.974174 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0208 04:10:42.974220 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:42.974228 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-3 ok I0208 04:10:42.974265 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:42.974332 3399 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0208 04:10:42.974365 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:42.974349 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0208 04:10:42.974398 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 } ] } I0208 04:10:42.974477 3399 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0208 04:10:42.974496 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0208 04:10:42.974530 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0208 04:10:42.974567 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } W0208 04:10:42.974478 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:42.974605 3399 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0208 04:10:42.974617 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } I0208 04:10:42.974618 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } security-group:sg-5 ok I0208 04:10:42.974651 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0208 04:10:42.974662 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0208 04:10:42.974680 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0208 04:10:42.974664 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0208 04:10:42.974701 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0208 04:10:42.974718 3399 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0208 04:10:42.974725 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:42.974775 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:42.974824 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0208 04:10:42.974928 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:42.974942 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0208 04:10:42.975015 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0208 04:10:42.975080 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:42.975152 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:42.975227 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:42.975490 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.88s) === RUN TestLifecyclePrivateKopeio I0208 04:10:42.975799 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:42.976022 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:42.976065 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:42.976108 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:42.976157 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:42.976197 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:42.976226 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:42.976264 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:42.976291 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:42.976344 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0208 04:10:43.006850 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.008964 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.009049 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.009103 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.046929 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.125907 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.127823 3399 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0208 04:10:43.128119 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0208 04:10:43.128149 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.128243 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0208 04:10:43.128357 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0208 04:10:43.130205 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0208 04:10:43.130081 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.130318 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:43.130318 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0208 04:10:43.130404 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.130469 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.130549 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0208 04:10:43.130631 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" }] } I0208 04:10:43.130582 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.130828 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.130852 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.131078 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.131289 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.131521 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:43.131605 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:43.132856 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.132903 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.132936 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.133057 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.133106 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.133219 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.133685 3399 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0208 04:10:43.133773 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.133865 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0208 04:10:43.133850 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0208 04:10:43.133934 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0208 04:10:43.133937 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0208 04:10:43.133971 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0208 04:10:43.133994 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0208 04:10:43.134064 3399 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0208 04:10:43.134075 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.134096 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0208 04:10:43.134109 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] } I0208 04:10:43.134171 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0208 04:10:43.134218 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.134245 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0208 04:10:43.134288 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0208 04:10:43.134302 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0208 04:10:43.134364 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0208 04:10:43.134394 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] }] } I0208 04:10:43.134549 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:43.134580 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.134659 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:43.134740 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0208 04:10:43.134726 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.134870 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0208 04:10:43.134890 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0208 04:10:43.135046 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:43.135098 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0208 04:10:43.135165 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] }] } I0208 04:10:43.135221 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.135274 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.135282 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:43.135323 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0208 04:10:43.135388 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.135442 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0208 04:10:43.135505 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.135442 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0208 04:10:43.135548 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:43.135615 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.135686 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.135717 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.135750 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.135783 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:43.135811 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:43.135836 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.135863 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.135886 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:43.135942 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:43.135633 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0208 04:10:43.135970 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:43.136027 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0208 04:10:43.136073 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:43.136106 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:43.136191 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:43.136241 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:43.136278 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.136356 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:43.136410 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0208 04:10:43.136457 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0208 04:10:43.136484 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:43.136572 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:43.136617 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:43.136660 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:43.136694 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:43.136735 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.136786 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0208 04:10:43.136829 3399 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0208 04:10:43.136907 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0208 04:10:43.138178 3399 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0208 04:10:43.138308 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0208 04:10:43.138395 3399 api.go:53] PageSize not implemented I0208 04:10:43.138394 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.138466 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } W0208 04:10:43.138512 3399 api.go:53] PageSize not implemented I0208 04:10:43.138549 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0208 04:10:43.138503 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0208 04:10:43.138611 3399 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0208 04:10:43.138653 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0208 04:10:43.138673 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0208 04:10:43.138708 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0208 04:10:43.138769 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.138701 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" }] }] } I0208 04:10:43.138808 3399 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0208 04:10:43.138841 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.138859 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0208 04:10:43.138887 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0208 04:10:43.138900 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.139013 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0208 04:10:43.139050 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0208 04:10:43.139130 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.139282 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.139521 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.139619 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0208 04:10:43.139646 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-2" } I0208 04:10:43.139678 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0208 04:10:43.139727 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.139820 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:43.139888 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.139934 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.140025 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.140072 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.140120 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0208 04:10:43.140177 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.140276 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0208 04:10:43.140378 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:43.140463 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" }] }] } I0208 04:10:43.140545 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.140625 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:43.140654 3399 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0208 04:10:43.140681 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:43.140705 3399 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0208 04:10:43.140731 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0208 04:10:43.140778 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0208 04:10:43.140805 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.140913 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0208 04:10:43.140992 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0208 04:10:43.141068 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0208 04:10:43.141104 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0208 04:10:43.141180 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0208 04:10:43.141213 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0208 04:10:43.141286 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.141374 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.141449 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.141515 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0208 04:10:43.141622 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" }] }] } I0208 04:10:43.141707 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:43.141766 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0208 04:10:43.141862 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:43.141895 3399 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0208 04:10:43.141920 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0208 04:10:43.141952 3399 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0208 04:10:43.141978 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.142026 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.142053 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:43.142131 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I0208 04:10:43.142228 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0208 04:10:43.142298 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.142325 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:43.142375 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0208 04:10:43.142405 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.142511 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0208 04:10:43.144289 3399 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0208 04:10:43.144395 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.144454 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.144576 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.144615 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.144726 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-2" } I0208 04:10:43.144786 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-1" } I0208 04:10:43.144853 3399 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0208 04:10:43.144968 3399 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0208 04:10:43.145013 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.145046 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.145100 3399 dns.go:238] Pre-creating DNS records I0208 04:10:43.145124 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.149324 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.151253 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.151311 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.151348 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.188962 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.289034 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.290499 3399 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0208 04:10:43.290850 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.290928 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.291008 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.291155 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.291239 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.291265 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.291313 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.291401 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.291487 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.291609 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.292925 3399 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0208 04:10:43.293088 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0208 04:10:43.293025 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.293208 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0208 04:10:43.293154 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0208 04:10:43.293258 3399 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0208 04:10:43.293329 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:43.293383 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0208 04:10:43.293516 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:43.293556 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.293664 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0208 04:10:43.293761 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.293794 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.293868 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0208 04:10:43.293952 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.293980 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0208 04:10:43.294060 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.294098 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.294151 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.294197 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.294280 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:43.294303 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.294372 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.294450 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.294473 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.294544 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0208 04:10:43.294565 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.294634 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:43.297690 3399 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0208 04:10:43.297808 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } W0208 04:10:43.297919 3399 api.go:53] PageSize not implemented I0208 04:10:43.297903 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.297944 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.298040 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.298008 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.298081 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } W0208 04:10:43.298143 3399 api.go:53] PageSize not implemented I0208 04:10:43.298179 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.298175 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.298286 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0208 04:10:43.298300 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.298338 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0208 04:10:43.298492 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0208 04:10:43.298568 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.298636 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.298711 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.298796 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.299550 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0208 04:10:43.299708 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.299882 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.300027 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:43.303023 3399 executor.go:111] Tasks: 117 done / 126 total; 6 can run W0208 04:10:43.303204 3399 api.go:53] PageSize not implemented I0208 04:10:43.303234 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.303158 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.303242 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.303332 3399 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0208 04:10:43.303334 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.303360 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0208 04:10:43.303416 3399 api.go:53] PageSize not implemented I0208 04:10:43.303434 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.303599 3399 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0208 04:10:43.303705 3399 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0208 04:10:43.307069 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:43.307129 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:43.307162 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:43.307199 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:43.307231 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:43.307267 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:43.307298 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:43.307329 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:43.307361 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:43.307395 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:43.307428 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:43.307464 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:43.307504 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0208 04:10:43.307545 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:43.307578 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:43.307609 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.307643 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:43.307675 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:43.307712 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0208 04:10:43.307747 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.307782 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.307815 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:43.307850 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:43.307884 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:43.307917 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.307952 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:43.307986 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:43.308022 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0208 04:10:43.308057 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0208 04:10:43.308092 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:43.308124 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:43.308155 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.308191 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.308227 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.308261 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:43.309875 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:43.309898 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:43.309937 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.309999 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0208 04:10:43.310071 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310130 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310176 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310224 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310262 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310314 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0208 04:10:43.310382 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310456 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:43.310493 3399 natgateway.go:117] DescribeNatGateways: { } I0208 04:10:43.310533 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0208 04:10:43.310575 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0208 04:10:43.310626 3399 api.go:53] PageSize not implemented I0208 04:10:43.310644 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0208 04:10:43.310695 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:43.310724 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:43.310741 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:43.310763 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:43.310780 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:43.310877 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.310904 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.310954 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:43.310993 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.311036 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:43.311083 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:43.311134 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-3","rtb-2"] } I0208 04:10:43.311193 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0208 04:10:43.311244 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0208 04:10:43.312325 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } autoscaling-group:bastion.privatekopeio.example.com ok I0208 04:10:43.312369 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0208 04:10:43.312409 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0208 04:10:43.312318 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } autoscaling-group:nodes.privatekopeio.example.com ok volume:vol-2 ok I0208 04:10:43.312457 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok I0208 04:10:43.312518 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0208 04:10:43.312476 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:43.312554 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0208 04:10:43.312574 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I0208 04:10:43.312630 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.312604 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0208 04:10:43.312674 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0208 04:10:43.312762 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.312722 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0208 04:10:43.312775 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } iam-instance-profile:nodes.privatekopeio.example.com ok I0208 04:10:43.312785 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok security-group:sg-3 ok I0208 04:10:43.312803 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.312849 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0208 04:10:43.312930 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.312942 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:43.312976 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:43.313010 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0208 04:10:43.313134 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.313150 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0208 04:10:43.313183 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:43.313226 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-1 ok I0208 04:10:43.313333 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.313342 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0208 04:10:43.313392 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0208 04:10:43.313403 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0208 04:10:43.313428 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0208 04:10:43.313455 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0208 04:10:43.313383 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0208 04:10:43.313499 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.313481 3399 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0208 04:10:43.313518 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:43.313532 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0208 04:10:43.313565 3399 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0208 04:10:43.313551 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0208 04:10:43.313597 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0208 04:10:43.313608 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:43.313632 3399 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0208 04:10:43.313655 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I0208 04:10:43.313663 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } subnet:subnet-3 ok I0208 04:10:43.313697 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0208 04:10:43.313712 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:43.313761 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0208 04:10:43.313819 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.313838 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0208 04:10:43.313873 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0208 04:10:43.313947 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0208 04:10:43.314007 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0208 04:10:43.314047 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:43.314106 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:43.314183 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:43.314513 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.34s) === RUN TestLifecycleIPv6 I0208 04:10:43.314823 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:43.315017 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:43.315055 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:43.315092 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:43.315136 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:43.315170 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:43.315199 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:43.315236 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:43.315261 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:43.315311 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0208 04:10:43.330944 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.335495 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.335570 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.372564 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.485946 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.487236 3399 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0208 04:10:43.487554 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.488613 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.488749 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.488838 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] }] } I0208 04:10:43.488992 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.489125 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.489273 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.489508 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.489539 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.489683 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:43.489636 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" }] } I0208 04:10:43.489774 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.489843 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" }] } I0208 04:10:43.490256 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.490767 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.490907 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.490975 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:43.490999 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:43.491022 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.491051 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.491359 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.491396 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.491536 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.492470 3399 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0208 04:10:43.492551 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.492623 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.492644 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0208 04:10:43.492657 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0208 04:10:43.492764 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.492836 3399 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0208 04:10:43.492794 3399 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0208 04:10:43.492864 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0208 04:10:43.492853 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.492941 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0208 04:10:43.493015 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.493031 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0208 04:10:43.493152 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.493198 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0208 04:10:43.493253 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:43.493210 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.493290 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0208 04:10:43.493435 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:43.493496 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.493526 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.493558 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0208 04:10:43.493674 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:43.493727 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.493751 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.493773 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:43.493796 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.493859 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:43.493936 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0208 04:10:43.493935 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:43.494037 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:43.494090 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.494126 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.494299 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.494380 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:43.494427 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:43.494465 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] } I0208 04:10:43.494332 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.495487 3399 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0208 04:10:43.495617 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.495693 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.495769 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0208 04:10:43.495817 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0208 04:10:43.495823 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0208 04:10:43.495919 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.495947 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.496044 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.496072 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I0208 04:10:43.496163 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0208 04:10:43.496288 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.496421 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:43.496481 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.496559 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0208 04:10:43.496634 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.496718 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.496771 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0208 04:10:43.496828 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.496883 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.496929 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.496989 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:43.497018 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.497016 3399 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497163 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497286 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497375 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497454 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497543 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497624 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497707 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:43.497742 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497826 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497898 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.497975 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0208 04:10:43.498054 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.498128 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.498209 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.498266 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.499378 3399 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0208 04:10:43.499450 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.499507 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:43.499560 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:43.499593 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:43.499644 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:43.499677 3399 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I0208 04:10:43.499699 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.499795 3399 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0208 04:10:43.499875 3399 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0208 04:10:43.499878 3399 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.500092 3399 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0208 04:10:43.500231 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:43.500301 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.500365 3399 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0208 04:10:43.500473 3399 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0208 04:10:43.500540 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.500558 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.500658 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0208 04:10:43.500732 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.500851 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.500917 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.500995 3399 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0208 04:10:43.501029 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.501054 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.501084 3399 dns.go:238] Pre-creating DNS records I0208 04:10:43.501098 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.504895 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.506894 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.506953 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.543644 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.684440 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.686622 3399 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0208 04:10:43.686853 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.686993 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.686984 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.687036 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.687140 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.687187 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.687330 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.687396 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.687510 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.687636 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.689079 3399 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0208 04:10:43.689201 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.689330 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.689337 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0208 04:10:43.689421 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0208 04:10:43.689396 3399 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0208 04:10:43.689436 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.689502 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0208 04:10:43.689515 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.689599 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0208 04:10:43.689700 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.689732 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0208 04:10:43.689814 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.689849 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0208 04:10:43.689941 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.689993 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.692851 3399 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0208 04:10:43.692942 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.693098 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.693138 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.693321 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:43.693405 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.693508 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.693601 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:43.693640 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0208 04:10:43.693674 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0208 04:10:43.693935 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.694043 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.694140 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.694276 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.694405 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.695847 3399 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0208 04:10:43.695910 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:43.695955 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:43.695930 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.695967 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0208 04:10:43.696036 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0208 04:10:43.696010 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.696073 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0208 04:10:43.696087 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:43.696112 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0208 04:10:43.696145 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0208 04:10:43.696258 3399 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0208 04:10:43.696326 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.696354 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.696404 3399 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0208 04:10:43.696426 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:43.696447 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0208 04:10:43.696479 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:43.696544 3399 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0208 04:10:43.696561 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:43.696578 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0208 04:10:43.696646 3399 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0208 04:10:43.698395 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:43.698433 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:43.698455 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:43.698475 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:43.698494 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:43.698516 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0208 04:10:43.698537 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:43.698559 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:43.698579 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:43.698599 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:43.698624 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:43.698646 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:43.698669 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:43.698694 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:43.698719 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:43.698739 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:43.698760 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.698782 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:43.698801 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:43.698822 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.698842 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.698863 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:43.698883 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.698907 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:43.698926 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:43.698945 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:43.698964 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.698985 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.699022 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.699046 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:43.699934 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:43.699951 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:43.699975 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700007 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0208 04:10:43.700045 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700079 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700111 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700143 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700168 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700196 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0208 04:10:43.700230 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700258 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:43.700276 3399 natgateway.go:117] DescribeNatGateways: { } I0208 04:10:43.700300 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0208 04:10:43.700325 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0208 04:10:43.700358 3399 api.go:53] PageSize not implemented I0208 04:10:43.700369 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:43.700384 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:43.700392 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0208 04:10:43.700414 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:43.700427 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:43.700437 3399 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3" ] } I0208 04:10:43.700475 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.700492 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.700519 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:43.700543 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.700571 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:43.700602 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:43.700649 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0208 04:10:43.700677 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:43.700699 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0208 04:10:43.701046 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:43.701069 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.701120 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0208 04:10:43.701127 3399 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0208 04:10:43.701105 3399 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } I0208 04:10:43.701053 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.minimal-ipv6.example.com ok route53-record:Z1AFAKE1ZON3YO/A/api.minimal-ipv6.example.com ok target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0208 04:10:43.701176 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:43.701166 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.701189 3399 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0208 04:10:43.701220 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok nat-gateway:nat-1 ok I0208 04:10:43.701229 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok I0208 04:10:43.701256 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0208 04:10:43.701345 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.701356 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0208 04:10:43.701401 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0208 04:10:43.701594 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:43.701615 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0208 04:10:43.701657 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:43.701705 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.701756 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:43.701792 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok autoscaling-config:lt-2 ok I0208 04:10:43.701865 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:43.701927 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0208 04:10:43.701956 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:43.702026 3399 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0208 04:10:43.702030 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.702063 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:43.702076 3399 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0208 04:10:43.702518 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0208 04:10:43.702569 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0208 04:10:43.702685 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.702895 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0208 04:10:43.702911 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0208 04:10:43.702952 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0208 04:10:43.703053 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:43.703143 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:43.703243 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:43.703607 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.39s) === RUN TestLifecycleSharedVPC I0208 04:10:43.703920 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:43.704198 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:43.704257 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:43.704308 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:43.704354 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:43.704812 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:43.704847 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:43.704875 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:43.704893 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:43.704937 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0208 04:10:43.720642 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.722449 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.722509 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.759358 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.841730 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.843430 3399 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0208 04:10:43.843755 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.843841 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0208 04:10:43.843963 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0208 04:10:43.845401 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0208 04:10:43.845513 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0208 04:10:43.843972 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.846447 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.846678 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:43.846781 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:43.846835 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.846930 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.847221 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.847531 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:43.847633 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.847682 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:43.847736 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.847895 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.849459 3399 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0208 04:10:43.849573 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0208 04:10:43.849611 3399 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0208 04:10:43.849618 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0208 04:10:43.849719 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0208 04:10:43.849803 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0208 04:10:43.849813 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:43.849937 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0208 04:10:43.850043 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0208 04:10:43.850135 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.850297 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.850301 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-12345678" } I0208 04:10:43.850373 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:43.850453 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0208 04:10:43.850478 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.850523 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.850564 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.850603 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0208 04:10:43.850665 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:43.850734 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.850776 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.850815 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }], VpcId: "vpc-12345678" } I0208 04:10:43.850871 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.850910 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.851519 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0208 04:10:43.854501 3399 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0208 04:10:43.854691 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.854823 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.854954 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.855034 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.855115 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.855190 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0208 04:10:43.855273 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.855367 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0208 04:10:43.855402 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.855569 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0208 04:10:43.855734 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.855921 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" }] }] } I0208 04:10:43.856125 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:43.856191 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" }] }] } I0208 04:10:43.856463 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.856716 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.856991 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:43.857055 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.857207 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.857344 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.857450 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.857542 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0208 04:10:43.857651 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" }] }] } I0208 04:10:43.857750 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.857824 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.860159 3399 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0208 04:10:43.860381 3399 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0208 04:10:43.860511 3399 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0208 04:10:43.860550 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.860593 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.860649 3399 dns.go:238] Pre-creating DNS records I0208 04:10:43.860678 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:43.866977 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:43.869965 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.870061 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.907352 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:43.987810 3399 iamrole.go:89] ListRoles: { } I0208 04:10:43.988799 3399 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0208 04:10:43.989020 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:43.989119 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:43.989141 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:43.989351 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:43.989294 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.989553 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.989644 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.989691 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:43.991212 3399 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0208 04:10:43.991361 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0208 04:10:43.991325 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0208 04:10:43.991447 3399 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0208 04:10:43.991504 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:43.991569 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.991697 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.991752 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:43.991867 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.992025 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0208 04:10:43.992180 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:43.994265 3399 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0208 04:10:43.994342 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.994425 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.994671 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.994754 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.994857 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:43.994935 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.995071 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:43.996334 3399 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0208 04:10:43.996506 3399 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0208 04:10:43.996582 3399 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0208 04:10:43.998263 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:43.998307 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:43.998326 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:43.998345 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:43.998364 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:43.998382 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:43.998400 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:43.998418 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:43.998440 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:43.998467 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:43.998494 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:43.998512 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:43.998530 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:43.998550 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:43.998566 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.998585 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:43.998604 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.998625 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:43.998643 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:43.998660 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:43.998677 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:43.998699 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:43.998721 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:43.999540 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:43.999558 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:43.999584 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999617 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0208 04:10:43.999651 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999686 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999711 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999734 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999758 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999782 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0208 04:10:43.999811 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999847 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0208 04:10:43.999869 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0208 04:10:43.999897 3399 api.go:53] PageSize not implemented I0208 04:10:43.999910 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:43.999926 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:43.999937 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:43.999951 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:43.999962 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:43.999992 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.000010 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:44.000038 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:44.000063 3399 iamrole.go:89] ListRoles: { } I0208 04:10:44.000089 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:44.000112 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:44.000129 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok I0208 04:10:44.000375 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:nodes.sharedvpc.example.com ok I0208 04:10:44.000443 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0208 04:10:44.000487 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } iam-instance-profile:masters.sharedvpc.example.com ok I0208 04:10:44.000505 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0208 04:10:44.000538 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0208 04:10:44.000427 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0208 04:10:44.000562 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:44.000572 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:44.000437 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedvpc.example.com. ok autoscaling-config:lt-2 ok I0208 04:10:44.000616 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0208 04:10:44.000686 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:44.000721 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:44.000753 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0208 04:10:44.000875 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:44.000888 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:44.000911 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:44.000969 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0208 04:10:44.000986 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:44.001000 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0208 04:10:44.001072 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0208 04:10:44.001092 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0208 04:10:44.001129 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0208 04:10:44.001167 3399 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0208 04:10:44.001183 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0208 04:10:44.001214 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0208 04:10:44.001600 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:44.001864 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.30s) === RUN TestLifecycleComplex I0208 04:10:44.002147 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:44.002390 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:44.002438 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:44.002488 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:44.002542 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:44.002588 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:44.002624 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:44.002666 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:44.002697 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:44.002760 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0208 04:10:44.039698 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:44.042372 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.042485 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.080205 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.210036 3399 iamrole.go:89] ListRoles: { } I0208 04:10:44.212467 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0208 04:10:44.212857 3399 api.go:53] PageSize not implemented I0208 04:10:44.212881 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.213028 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:44.213033 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.213049 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0208 04:10:44.214156 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.214248 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0208 04:10:44.214543 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:44.214761 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:44.215211 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:44.215238 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0208 04:10:44.215321 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0208 04:10:44.215291 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0208 04:10:44.215334 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.215484 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:44.215592 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:44.215921 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:44.216010 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:44.216932 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" } ] }] } I0208 04:10:44.217064 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:44.217102 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:44.217155 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:44.217195 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:44.217234 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:44.217274 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.219097 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:44.219213 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.219287 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:44.219324 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:44.219322 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0208 04:10:44.219322 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:44.219404 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:44.219399 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0208 04:10:44.219456 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0208 04:10:44.219528 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:44.219446 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:44.219476 3399 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0208 04:10:44.219562 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:44.219594 3399 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0208 04:10:44.219651 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.219863 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0208 04:10:44.219884 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.220067 3399 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0208 04:10:44.220125 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.220321 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0208 04:10:44.220303 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:44.220547 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.220742 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.220756 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:44.220920 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:44.220679 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0208 04:10:44.220967 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.221075 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.221163 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0208 04:10:44.221168 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0208 04:10:44.221259 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.221330 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0208 04:10:44.221442 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0208 04:10:44.221518 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0208 04:10:44.221598 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0208 04:10:44.221669 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:44.221715 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:44.221744 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:44.221778 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0208 04:10:44.221833 3399 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0208 04:10:44.221862 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0208 04:10:44.221951 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0208 04:10:44.222006 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0208 04:10:44.222068 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:44.222107 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:44.222145 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" } ] } I0208 04:10:44.222242 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:44.222319 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:44.222380 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:44.222428 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:44.222470 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:44.222505 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:44.222566 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:44.222597 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:44.222627 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:44.222652 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:44.222775 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:44.222899 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0208 04:10:44.222999 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.223022 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:44.223105 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:44.223174 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0208 04:10:44.223226 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0208 04:10:44.223216 3399 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], Type: "network" } I0208 04:10:44.223338 3399 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0208 04:10:44.223276 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I0208 04:10:44.223400 3399 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0208 04:10:44.223446 3399 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.223530 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0208 04:10:44.223721 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0208 04:10:44.223814 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0208 04:10:44.223971 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0208 04:10:44.224065 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.224120 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.224165 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.224207 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.224282 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.224354 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0208 04:10:44.224620 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0208 04:10:44.224786 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.224878 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.224952 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0208 04:10:44.225135 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0208 04:10:44.225232 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0208 04:10:44.225329 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:44.225381 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0208 04:10:44.225426 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I0208 04:10:44.225584 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0208 04:10:44.225631 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0208 04:10:44.225689 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0208 04:10:44.225787 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0208 04:10:44.225940 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0208 04:10:44.226033 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0208 04:10:44.226196 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0208 04:10:44.226355 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0208 04:10:44.226516 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.226596 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0208 04:10:44.226697 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0208 04:10:44.226792 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:44.226847 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0208 04:10:44.226997 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0208 04:10:44.227149 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" } ] }] } I0208 04:10:44.227314 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0208 04:10:44.227533 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0208 04:10:44.227767 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0208 04:10:44.227931 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.228013 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:44.228074 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0208 04:10:44.228126 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.228194 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0208 04:10:44.228419 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0208 04:10:44.231685 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:44.231758 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:44.231917 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:44.232027 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:44.232116 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0208 04:10:44.232156 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.232186 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:44.232236 3399 dns.go:238] Pre-creating DNS records I0208 04:10:44.232260 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:44.240923 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:44.245003 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.245093 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.281987 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.439540 3399 iamrole.go:89] ListRoles: { } I0208 04:10:44.441072 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0208 04:10:44.441227 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0208 04:10:44.441336 3399 api.go:53] PageSize not implemented I0208 04:10:44.441372 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:44.441454 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.441584 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:44.441602 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:44.441771 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:44.441873 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.442002 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:44.442036 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:44.442101 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.442147 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.442279 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:44.444333 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:44.444449 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.444519 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:44.444495 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:44.444595 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:44.444577 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.444550 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.444696 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:44.444778 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.444692 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.444882 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.444944 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.444994 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.445079 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.445175 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:44.445202 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:44.445303 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:44.445342 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:44.445456 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:44.445585 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0208 04:10:44.445744 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:44.445784 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:44.445944 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:44.445993 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.446102 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0208 04:10:44.446256 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:44.449580 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:44.449732 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.449831 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.449842 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:44.449923 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:44.449945 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.450000 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.449977 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.450040 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.450081 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.450122 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.450155 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.450247 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.450323 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.450528 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.450622 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.450775 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.451045 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.451176 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.451300 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.454902 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:44.455239 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:44.455347 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:44.455394 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:44.456992 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.457112 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:44.457247 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0208 04:10:44.464033 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.468057 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.482258 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:44.485134 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.485207 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.523107 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.678468 3399 iamrole.go:89] ListRoles: { } I0208 04:10:44.680723 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0208 04:10:44.680984 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.681142 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.681302 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:44.681334 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:44.681398 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.681437 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:44.681490 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:44.681543 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0208 04:10:44.681661 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } W0208 04:10:44.681757 3399 api.go:53] PageSize not implemented I0208 04:10:44.681752 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.681906 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:44.684458 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:44.687714 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:44.687841 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:44.687994 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.688121 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:44.688225 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.687938 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:44.688418 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:44.687848 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:44.689247 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:44.689324 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.689413 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.689486 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.689607 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.689718 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.689830 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.689929 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.690006 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:44.690124 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.690223 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.690303 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:44.690418 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.690512 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:44.690554 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:44.690639 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:44.690682 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:44.690805 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:44.691635 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:44.691788 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:44.691907 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:44.691932 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.691799 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.691974 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.692012 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.692049 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.692085 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.692059 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.692115 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.692212 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.692274 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0208 04:10:44.692310 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.692300 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.692347 3399 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0208 04:10:44.692383 3399 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0208 04:10:44.692418 3399 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0208 04:10:44.692497 3399 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0208 04:10:44.692553 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.692585 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.692843 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.693025 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.693222 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.693371 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.693516 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.693634 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.693803 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.696703 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:44.696787 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:44.696852 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:44.696870 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:44.696888 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.696982 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:44.697109 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0208 04:10:44.697152 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.697191 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:44.697253 3399 dns.go:238] Pre-creating DNS records I0208 04:10:44.705815 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:44.709978 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.710079 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.746820 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.930689 3399 iamrole.go:89] ListRoles: { } I0208 04:10:44.933078 3399 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0208 04:10:44.933349 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:44.933559 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } W0208 04:10:44.933666 3399 api.go:53] PageSize not implemented I0208 04:10:44.933635 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:44.933748 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:44.933872 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.934021 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.934135 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:44.934173 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:44.934214 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:44.934358 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.934513 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:44.934613 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.935418 3399 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0208 04:10:44.935551 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:44.935595 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:44.935551 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.935649 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:44.935643 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.935726 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:44.935776 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.935800 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.935829 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.935861 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.935912 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.935987 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:44.936090 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:44.936129 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.936217 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:44.936241 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.936293 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:44.936339 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:44.936419 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:44.936446 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:44.936529 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:44.936573 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.936652 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:44.936674 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.936742 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:44.940502 3399 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0208 04:10:44.940653 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:44.940725 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:44.940746 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.940732 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.940789 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.940833 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.940827 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.940873 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:44.940905 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:44.940934 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:44.941142 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.941256 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.941430 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.942168 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:44.942244 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.942428 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.942489 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:44.942538 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.942639 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.943887 3399 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0208 04:10:44.943998 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:44.944054 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:44.944069 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:44.944085 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:44.944138 3399 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0208 04:10:44.944246 3399 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0208 04:10:44.948095 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.952321 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:44.964573 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:44.967359 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:44.967455 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.004952 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.142672 3399 iamrole.go:89] ListRoles: { } I0208 04:10:45.144124 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0208 04:10:45.144350 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0208 04:10:45.144549 3399 api.go:53] PageSize not implemented I0208 04:10:45.144501 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.144546 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:45.144651 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:45.144783 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.144921 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:45.144959 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:45.145031 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.145075 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:45.145218 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.145361 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:45.145468 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.145611 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:45.146257 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:45.146336 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:45.146406 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:45.146436 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:45.146422 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.146475 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:45.146515 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.146561 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.146607 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:45.146629 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:45.146686 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.146634 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.146800 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:45.146845 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.146926 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:45.147064 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:45.147123 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.147238 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:45.147284 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.147355 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:45.147482 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:45.147548 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.147672 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.147792 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:45.147832 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.147898 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.150335 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:45.150467 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.150497 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:45.150553 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:45.150569 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.150612 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.150584 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.150643 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.150680 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.150705 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.150726 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.150698 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.150834 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.150869 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.150973 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.151075 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.151356 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.151705 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.151880 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.152054 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.152688 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.152732 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.152838 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:45.152912 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:45.152971 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:45.152998 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:45.153021 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.153085 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:45.153191 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:45.153235 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.153268 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:45.153319 3399 dns.go:238] Pre-creating DNS records I0208 04:10:45.161259 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:45.165966 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.166093 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.202952 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.349926 3399 iamrole.go:89] ListRoles: { } I0208 04:10:45.351341 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0208 04:10:45.351493 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.351590 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } W0208 04:10:45.351618 3399 api.go:53] PageSize not implemented I0208 04:10:45.351608 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:45.351731 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:45.351853 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:45.351863 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:45.351984 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.352085 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.352113 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:45.352179 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.352260 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:45.352316 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:45.352399 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.354889 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:45.355046 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:45.355224 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:45.355226 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.355432 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:45.355348 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.355415 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.355487 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:45.355499 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.355574 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.355569 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:45.355632 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.355689 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.356315 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:45.356476 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:45.356519 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.357022 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:45.357064 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.357140 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:45.357258 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0208 04:10:45.357367 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.357471 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.357566 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:45.357721 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:45.357783 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:45.357811 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:45.359417 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:45.359515 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.359607 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.359686 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:45.359752 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:45.359764 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.359794 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.359816 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.359815 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.359886 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.359841 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.359938 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.359938 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.359961 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.359983 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.360112 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.360260 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.360377 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.360778 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.360882 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.362275 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:45.362342 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:45.362396 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:45.362413 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:45.362427 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.362473 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:45.362558 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0208 04:10:45.365981 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.368156 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.368222 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.374109 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:45.376941 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.377006 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.377045 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.416077 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.570565 3399 iamrole.go:89] ListRoles: { } I0208 04:10:45.571929 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0208 04:10:45.572081 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:45.572154 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:45.572312 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0208 04:10:45.572467 3399 api.go:53] PageSize not implemented I0208 04:10:45.572489 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.572634 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.572724 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:45.572748 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:45.572785 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.572829 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:45.572911 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.573000 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:45.573070 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:45.573139 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:45.574164 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:45.574305 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:45.574316 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.574308 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:45.574376 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.574447 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.574470 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:45.574480 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.574509 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:45.574560 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.574564 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.574622 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.574672 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.574723 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:45.574820 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:45.574865 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.574954 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:45.574979 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:45.575072 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:45.575124 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:45.575285 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:45.575349 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.575470 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0208 04:10:45.575614 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:45.575653 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.575964 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:45.577448 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:45.577569 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.577655 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:45.577728 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:45.577711 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.577745 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.577779 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.577802 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.577826 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.577822 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.577846 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.577865 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.577959 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.578067 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.578399 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.578545 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.578674 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.578781 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.578842 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.578982 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.580713 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:45.580781 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:45.580849 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:45.580866 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:45.580881 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.580997 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:45.581104 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:45.581143 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.581171 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:45.581215 3399 dns.go:238] Pre-creating DNS records I0208 04:10:45.587072 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:45.590679 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.590799 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.590875 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.627780 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.747526 3399 iamrole.go:89] ListRoles: { } I0208 04:10:45.748925 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0208 04:10:45.749077 3399 api.go:53] PageSize not implemented I0208 04:10:45.749160 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.749308 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.749341 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:45.749453 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:45.749549 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.749677 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:45.749764 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:45.749840 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.749923 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:45.749945 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:45.749988 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.750031 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0208 04:10:45.750095 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:45.751226 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:45.751644 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.751769 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:45.751758 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:45.751746 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.751814 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.751830 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:45.751862 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:45.751859 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.751903 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.751947 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.752098 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:45.752143 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:45.752293 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:45.752360 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.752491 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:45.752536 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.752647 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.752797 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.752892 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:45.753746 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:45.753828 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:45.753995 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.754122 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.754367 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:45.754745 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:45.757201 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:45.757318 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.757465 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:45.757523 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:45.757536 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.757562 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.757583 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.757608 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.757630 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.757600 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.757649 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.757870 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.758107 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.758195 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.758314 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.758669 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.758864 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.759043 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.759233 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.759418 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.762884 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:45.763004 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:45.763097 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:45.763120 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:45.763143 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.763222 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:45.763345 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0208 04:10:45.766534 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.769727 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.769822 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.769962 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.779340 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:45.782742 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.782822 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.782860 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.782891 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.820425 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.958266 3399 iamrole.go:89] ListRoles: { } I0208 04:10:45.961197 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0208 04:10:45.961366 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:45.961488 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0208 04:10:45.961575 3399 api.go:53] PageSize not implemented I0208 04:10:45.961534 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:45.961557 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.961664 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:45.961740 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.961877 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:45.961958 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.962043 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:45.962131 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0208 04:10:45.962223 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:45.962252 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:45.962311 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.963335 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:45.963470 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.963567 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:45.963613 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:45.963622 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.963596 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.963688 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:45.963684 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.963708 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:45.963753 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:45.963763 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.963785 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.963875 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:45.963903 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:45.964035 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:45.964075 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.964165 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:45.964268 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:45.964308 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.964385 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.964457 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:45.964541 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:45.964586 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:45.964675 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:45.964701 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.964756 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:45.966597 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:45.966740 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:45.966834 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:45.966853 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.966899 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.966746 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.966936 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.967003 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:45.967044 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:45.967032 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.967072 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:45.967238 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.967357 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.967477 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.967725 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.967912 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.968033 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:45.968155 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.968275 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:45.968365 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.971224 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:45.971322 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:45.971406 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:45.971435 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:45.971466 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:45.971546 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:45.971653 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:45.971695 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.971735 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:45.971790 3399 dns.go:238] Pre-creating DNS records I0208 04:10:45.980365 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0208 04:10:45.984588 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.984690 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.984750 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:45.984814 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.021839 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.153857 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.155262 3399 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0208 04:10:46.155419 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0208 04:10:46.155558 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0208 04:10:46.155712 3399 api.go:53] PageSize not implemented I0208 04:10:46.155638 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.155773 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:46.155802 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:46.155728 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.155837 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.155956 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0208 04:10:46.156031 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.156126 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0208 04:10:46.156185 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.156211 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0208 04:10:46.156271 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.157582 3399 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0208 04:10:46.157713 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0208 04:10:46.157710 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.157787 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:46.157806 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.157863 3399 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:46.157774 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:46.157893 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.157916 3399 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0208 04:10:46.157991 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:46.157994 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.158081 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0208 04:10:46.158106 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.158157 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.158202 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0208 04:10:46.158292 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.158338 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.158441 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0208 04:10:46.158545 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:46.158589 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.158663 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0208 04:10:46.158775 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.158819 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.158977 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:46.159031 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:46.159067 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.163028 3399 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0208 04:10:46.163252 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:46.163326 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0208 04:10:46.163327 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0208 04:10:46.163384 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:46.163425 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.163471 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:46.163500 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:46.163521 3399 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:46.163544 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0208 04:10:46.163563 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0208 04:10:46.163582 3399 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0208 04:10:46.163930 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.164035 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.164168 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.164308 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.164382 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:46.164458 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.164561 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.164704 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.167586 3399 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0208 04:10:46.167669 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:46.167727 3399 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0208 04:10:46.167743 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0208 04:10:46.167759 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:46.167854 3399 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0208 04:10:46.167966 3399 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0208 04:10:46.168234 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:46.168270 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:46.168291 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:46.168313 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:46.168334 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:46.168358 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:46.168377 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:46.168397 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:46.168415 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:46.168437 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:46.168460 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:46.168486 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:46.168507 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:46.168526 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:46.168551 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:46.168571 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:46.168594 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.168616 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.168639 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:46.168661 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:46.168684 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:46.168704 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:46.168728 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0208 04:10:46.168751 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:46.168772 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:46.168797 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.168821 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.168843 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:46.168866 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:46.170476 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:46.170497 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:46.170518 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170557 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0208 04:10:46.170595 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170632 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170660 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170696 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170723 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170752 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0208 04:10:46.170788 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170833 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0208 04:10:46.170860 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0208 04:10:46.170892 3399 api.go:53] PageSize not implemented I0208 04:10:46.170904 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:46.170919 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:46.170927 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0208 04:10:46.170952 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:46.170966 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:46.170977 3399 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0208 04:10:46.171020 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.171038 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.171065 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:46.171087 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.171113 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:46.171142 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:46.171171 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0208 04:10:46.171507 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:46.171553 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-group:nodes.complex.example.com ok I0208 04:10:46.171536 3399 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0208 04:10:46.171590 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } I0208 04:10:46.171643 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok autoscaling-config:lt-1 ok I0208 04:10:46.171601 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0208 04:10:46.171661 3399 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0208 04:10:46.171702 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0208 04:10:46.171709 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0208 04:10:46.171739 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0208 04:10:46.171741 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0208 04:10:46.171767 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0208 04:10:46.171843 3399 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok volume:vol-2 ok iam-instance-profile:nodes.complex.example.com ok I0208 04:10:46.171802 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W0208 04:10:46.172035 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:46.172051 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:46.171923 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I0208 04:10:46.172083 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:46.172139 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok autoscaling-config:lt-2 ok I0208 04:10:46.172212 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:46.172268 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.172313 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 } ] } W0208 04:10:46.172686 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:46.172701 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:46.172785 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I0208 04:10:46.172794 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } subnet:subnet-2 ok I0208 04:10:46.172842 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0208 04:10:46.172857 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0208 04:10:46.172889 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0208 04:10:46.172919 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0208 04:10:46.172847 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0208 04:10:46.172958 3399 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0208 04:10:46.172992 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0208 04:10:46.173025 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0208 04:10:46.173074 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:46.173128 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:46.173174 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:46.173371 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.17s) === RUN TestLifecycleExternalLB I0208 04:10:46.173606 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:46.173767 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:46.173795 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:46.173820 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:46.173853 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:46.173880 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:46.173897 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:46.173920 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:46.173937 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:46.173974 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0208 04:10:46.189019 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.190819 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.190878 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.227630 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.316085 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.317174 3399 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0208 04:10:46.317318 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.317358 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0208 04:10:46.317444 3399 api.go:53] PageSize not implemented I0208 04:10:46.317459 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.317451 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.317523 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } W0208 04:10:46.317564 3399 api.go:53] PageSize not implemented I0208 04:10:46.317511 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0208 04:10:46.317582 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.317608 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.317636 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0208 04:10:46.317708 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.317705 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } W0208 04:10:46.317796 3399 api.go:53] PageSize not implemented I0208 04:10:46.317796 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.317764 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0208 04:10:46.317807 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0208 04:10:46.317850 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0208 04:10:46.317838 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.317881 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.317917 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.317904 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0208 04:10:46.317966 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.317978 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.318017 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.318086 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.318265 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.318295 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:46.318396 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.318431 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.318538 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:46.318561 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:46.318578 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:46.318604 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.319585 3399 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0208 04:10:46.319692 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0208 04:10:46.319781 3399 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0208 04:10:46.319817 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0208 04:10:46.319760 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0208 04:10:46.319873 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.319932 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0208 04:10:46.320009 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.320147 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.320210 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:46.320249 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0208 04:10:46.320241 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0208 04:10:46.320377 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:46.320439 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.320467 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.320493 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.320618 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0208 04:10:46.320632 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.320796 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.320867 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:46.320947 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0208 04:10:46.320977 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:46.321050 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.321084 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.321109 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:46.321157 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:46.321190 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:46.321216 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:46.321248 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:46.321276 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0208 04:10:46.321148 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0208 04:10:46.324499 3399 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0208 04:10:46.324700 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.324796 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.324904 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:46.325380 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0208 04:10:46.325529 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.325681 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.325833 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0208 04:10:46.325926 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:46.325955 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.326001 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.326076 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.326180 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.326258 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.326337 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.326527 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.326775 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" }] }] } I0208 04:10:46.326940 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:46.326998 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0208 04:10:46.327103 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0208 04:10:46.327178 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0208 04:10:46.327257 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:46.327291 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" }] }] } I0208 04:10:46.327374 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:46.327452 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.327510 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.328153 3399 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0208 04:10:46.328325 3399 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0208 04:10:46.328412 3399 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0208 04:10:46.328441 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.328466 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.328501 3399 dns.go:238] Pre-creating DNS records I0208 04:10:46.328518 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.332126 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.333889 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.333952 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.371191 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.469164 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.470561 3399 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0208 04:10:46.470720 3399 api.go:53] PageSize not implemented I0208 04:10:46.470881 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0208 04:10:46.471073 3399 api.go:53] PageSize not implemented I0208 04:10:46.471072 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0208 04:10:46.471160 3399 api.go:53] PageSize not implemented I0208 04:10:46.471012 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.471164 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.471229 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.471189 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.472268 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.472477 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:46.472521 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:46.472630 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.472707 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.472861 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.472871 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.473129 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.473253 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.473380 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0208 04:10:46.473528 3399 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0208 04:10:46.473652 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0208 04:10:46.476051 3399 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0208 04:10:46.476136 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0208 04:10:46.476227 3399 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0208 04:10:46.476204 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.476297 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:46.476366 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.476502 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:46.476542 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.476657 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0208 04:10:46.476760 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.476799 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0208 04:10:46.476904 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.476969 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.481692 3399 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0208 04:10:46.481831 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.481912 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.481996 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.482107 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.482183 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.482314 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.482791 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.483922 3399 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0208 04:10:46.484080 3399 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0208 04:10:46.484169 3399 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0208 04:10:46.486199 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:46.486239 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:46.486261 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:46.486286 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:46.486308 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:46.486330 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:46.486351 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:46.486370 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:46.486388 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:46.486412 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:46.486433 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:46.486459 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:46.486483 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:46.486503 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:46.486522 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:46.486543 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:46.486562 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.486582 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.486602 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:46.486625 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:46.486644 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:46.486664 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:46.486682 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.486706 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.486727 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:46.486748 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:46.487582 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:46.487599 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:46.487622 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487652 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0208 04:10:46.487686 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487724 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487753 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487785 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487810 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487834 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0208 04:10:46.487863 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487896 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0208 04:10:46.487921 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0208 04:10:46.487956 3399 api.go:53] PageSize not implemented I0208 04:10:46.487971 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:46.487986 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:46.487998 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:46.488012 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:46.488023 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.488057 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.488074 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.488106 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:46.488129 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.488155 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:46.488185 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:46.488207 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:46.488475 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0208 04:10:46.488517 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0208 04:10:46.488600 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-group:nodes.externallb.example.com ok I0208 04:10:46.488550 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0208 04:10:46.488615 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:46.488647 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0208 04:10:46.488666 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:46.488693 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:46.488691 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0208 04:10:46.488730 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.488743 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0208 04:10:46.488774 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0208 04:10:46.488917 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:46.488930 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:46.488949 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:46.488678 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.488989 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.externallb.example.com. ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0208 04:10:46.489095 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:46.489245 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:46.489240 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0208 04:10:46.489310 3399 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0208 04:10:46.489324 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0208 04:10:46.489347 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0208 04:10:46.489368 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0208 04:10:46.489397 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0208 04:10:46.489404 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0208 04:10:46.489420 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0208 04:10:46.489457 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:46.489497 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:46.489538 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:46.489712 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.32s) === RUN TestLifecycleSharedSubnet I0208 04:10:46.489890 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:46.490069 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:46.490097 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:46.490124 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:46.490156 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:46.490182 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:46.490199 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:46.490223 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:46.490239 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:46.490274 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0208 04:10:46.506032 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.507783 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.507840 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.545417 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.611675 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.612552 3399 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0208 04:10:46.612769 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:46.612852 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:46.612875 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:46.612978 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.613021 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0208 04:10:46.613109 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:46.613114 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0208 04:10:46.613205 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.613162 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.613237 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.613312 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0208 04:10:46.613340 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:46.613376 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0208 04:10:46.613409 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.613439 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.613471 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.613556 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0208 04:10:46.615139 3399 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0208 04:10:46.615307 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0208 04:10:46.615262 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:46.615366 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0208 04:10:46.615429 3399 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0208 04:10:46.615486 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0208 04:10:46.615423 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0208 04:10:46.615719 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0208 04:10:46.615934 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.616019 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.616063 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.616095 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0208 04:10:46.616075 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0208 04:10:46.616246 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.616357 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.616401 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.616435 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:46.616471 3399 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" } ] } I0208 04:10:46.616536 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0208 04:10:46.619330 3399 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0208 04:10:46.619484 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I0208 04:10:46.619661 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0208 04:10:46.620634 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" }] }] } I0208 04:10:46.620859 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.621028 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.621188 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.621317 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.621562 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" }] }] } I0208 04:10:46.621798 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.621955 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.622134 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.622951 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.623437 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.623555 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" }] }] } I0208 04:10:46.623637 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.623716 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.623791 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.623938 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0208 04:10:46.625548 3399 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0208 04:10:46.625768 3399 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0208 04:10:46.625921 3399 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0208 04:10:46.625965 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.625994 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.626041 3399 dns.go:238] Pre-creating DNS records I0208 04:10:46.626066 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.631751 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.633991 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.634055 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.670197 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.733811 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.734796 3399 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0208 04:10:46.734971 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.735127 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:46.735177 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:46.735195 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:46.735187 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.735228 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.735396 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.735459 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.737685 3399 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0208 04:10:46.737963 3399 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0208 04:10:46.737838 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0208 04:10:46.738029 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0208 04:10:46.738111 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.738227 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0208 04:10:46.738383 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0208 04:10:46.738643 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:46.738783 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.742341 3399 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0208 04:10:46.742451 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.742568 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.742732 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.742824 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:46.743847 3399 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0208 04:10:46.743998 3399 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0208 04:10:46.744084 3399 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0208 04:10:46.745780 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:46.745816 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:46.745837 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:46.745855 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:46.745873 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:46.745892 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:46.745910 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:46.745928 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:46.745948 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:46.745969 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:46.745991 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:46.746010 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:46.746026 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:46.746044 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.746065 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.746085 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:46.746107 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:46.746125 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:46.746144 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.746165 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.746185 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:46.747186 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:46.747210 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:46.747245 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747288 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0208 04:10:46.747341 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747397 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747432 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747472 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747517 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747555 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0208 04:10:46.747600 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747631 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0208 04:10:46.747652 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0208 04:10:46.747680 3399 api.go:53] PageSize not implemented I0208 04:10:46.747694 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:46.747711 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:46.747724 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:46.747740 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:46.747751 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:46.747780 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.747798 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.747829 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:46.747852 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.747880 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:46.747908 3399 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-2 ok autoscaling-group:nodes.sharedsubnet.example.com ok I0208 04:10:46.748126 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-config:lt-1 ok I0208 04:10:46.748179 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } keypair:key-1 ok I0208 04:10:46.748233 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0208 04:10:46.748172 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.748300 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0208 04:10:46.748340 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0208 04:10:46.748289 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0208 04:10:46.748384 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:46.748395 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:46.748274 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I0208 04:10:46.748421 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0208 04:10:46.748456 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.748493 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0208 04:10:46.748615 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:46.748629 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:46.748650 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:46.748705 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0208 04:10:46.748741 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0208 04:10:46.748751 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0208 04:10:46.748771 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0208 04:10:46.748791 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0208 04:10:46.748814 3399 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0208 04:10:46.748821 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0208 04:10:46.748837 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0208 04:10:46.749030 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.26s) === RUN TestLifecyclePrivateSharedSubnet I0208 04:10:46.749233 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:46.749410 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:46.749438 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:46.749472 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:46.749501 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:46.749528 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:46.749544 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:46.749566 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:46.749582 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:46.749618 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0208 04:10:46.766758 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.768644 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.768698 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.768732 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.805257 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.872787 3399 iamrole.go:89] ListRoles: { } I0208 04:10:46.874118 3399 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0208 04:10:46.874311 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0208 04:10:46.874289 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:46.874386 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0208 04:10:46.874485 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:46.874537 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:46.874554 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:46.874582 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:46.874617 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0208 04:10:46.874649 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:46.874706 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0208 04:10:46.874670 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:46.874835 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:46.874970 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0208 04:10:46.875027 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.875091 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" }] } I0208 04:10:46.875127 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:46.875247 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:46.875415 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:46.875472 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.877750 3399 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0208 04:10:46.878054 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0208 04:10:46.877967 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0208 04:10:46.878216 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0208 04:10:46.878328 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0208 04:10:46.878385 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-subnet.example.com" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.878566 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:46.878576 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0208 04:10:46.878658 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:46.878769 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:46.878819 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0208 04:10:46.878944 3399 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0208 04:10:46.878940 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0208 04:10:46.879171 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0208 04:10:46.879092 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0208 04:10:46.879416 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0208 04:10:46.879403 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:46.879496 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0208 04:10:46.879666 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0208 04:10:46.879824 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0208 04:10:46.879948 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0208 04:10:46.880062 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:46.880115 3399 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] } I0208 04:10:46.880198 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.880324 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:46.880380 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:46.880432 3399 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] } I0208 04:10:46.880530 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.880627 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:46.880673 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:46.880718 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.880816 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:46.880865 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:46.880912 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:46.880996 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:46.881039 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:46.881522 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0208 04:10:46.884961 3399 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0208 04:10:46.885160 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } W0208 04:10:46.885351 3399 api.go:53] PageSize not implemented W0208 04:10:46.885493 3399 api.go:53] PageSize not implemented I0208 04:10:46.885335 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.886378 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0208 04:10:46.886380 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.886552 3399 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0208 04:10:46.886697 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0208 04:10:46.886768 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0208 04:10:46.886880 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0208 04:10:46.886797 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:46.887038 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0208 04:10:46.887041 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.887134 3399 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0208 04:10:46.887178 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.887207 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0208 04:10:46.887250 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0208 04:10:46.887301 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0208 04:10:46.887262 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.887435 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:46.887709 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" }] }] } I0208 04:10:46.887893 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.888063 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.888195 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.888309 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.888385 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.888430 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0208 04:10:46.888543 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.888693 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.888786 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.888876 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:46.888932 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.889026 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.889111 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.889204 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.889324 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0208 04:10:46.889408 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.889541 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.889637 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.889700 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0208 04:10:46.889855 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0208 04:10:46.889980 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0208 04:10:46.890583 3399 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0208 04:10:46.890689 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:46.890789 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:46.890875 3399 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0208 04:10:46.890994 3399 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0208 04:10:46.891054 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:46.891092 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.891151 3399 dns.go:238] Pre-creating DNS records I0208 04:10:46.891181 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:46.895874 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:46.898169 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.898242 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.898306 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:46.934266 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.000013 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.001809 3399 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0208 04:10:47.001946 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:47.002072 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:47.002115 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:47.002132 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:47.002163 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.002317 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.004062 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.004156 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.007137 3399 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0208 04:10:47.007226 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0208 04:10:47.007346 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.007405 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0208 04:10:47.007383 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0208 04:10:47.007435 3399 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0208 04:10:47.007485 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0208 04:10:47.007457 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0208 04:10:47.007579 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:47.007615 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:47.007676 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0208 04:10:47.007738 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0208 04:10:47.007793 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0208 04:10:47.007878 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:47.007913 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.007948 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0208 04:10:47.008026 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:47.010348 3399 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0208 04:10:47.010591 3399 api.go:53] PageSize not implemented I0208 04:10:47.010577 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.010622 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.010706 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0208 04:10:47.010750 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0208 04:10:47.010691 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0208 04:10:47.010804 3399 api.go:53] PageSize not implemented I0208 04:10:47.010914 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.010987 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.011024 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0208 04:10:47.011267 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.011421 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.011580 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.011701 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.013043 3399 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0208 04:10:47.013093 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } W0208 04:10:47.013129 3399 api.go:53] PageSize not implemented I0208 04:10:47.013142 3399 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0208 04:10:47.013149 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.013270 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0208 04:10:47.013306 3399 api.go:53] PageSize not implemented I0208 04:10:47.013316 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.013427 3399 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0208 04:10:47.013541 3399 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0208 04:10:47.015624 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:47.015659 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:47.015691 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:47.015710 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:47.015732 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.015752 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:47.015771 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:47.015789 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:47.015811 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:47.015833 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:47.015857 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0208 04:10:47.015887 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:47.015907 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:47.015926 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:47.015944 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:47.015965 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:47.015987 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:47.016007 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:47.016027 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:47.016047 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:47.016069 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:47.016089 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:47.016108 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:47.016129 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:47.016151 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:47.017081 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:47.017096 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:47.017119 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017159 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0208 04:10:47.017205 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017239 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017264 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017289 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017314 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017335 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0208 04:10:47.017358 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017387 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0208 04:10:47.017408 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0208 04:10:47.017435 3399 api.go:53] PageSize not implemented I0208 04:10:47.017445 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0208 04:10:47.017477 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:47.017493 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:47.017505 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:47.017520 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:47.017532 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:47.017561 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.017578 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.017608 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:47.017633 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.017662 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:47.017687 3399 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.private-shared-subnet.example.com ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0208 04:10:47.017979 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0208 04:10:47.018035 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0208 04:10:47.018000 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } load-balancer:api-private-shared-subnet-n2f8ak ok I0208 04:10:47.018093 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0208 04:10:47.018103 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok I0208 04:10:47.018059 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok I0208 04:10:47.018054 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0208 04:10:47.018136 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } W0208 04:10:47.018199 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.018201 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0208 04:10:47.018222 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0208 04:10:47.018227 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } autoscaling-config:lt-1 ok I0208 04:10:47.018255 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0208 04:10:47.018275 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0208 04:10:47.018352 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.018400 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0208 04:10:47.018530 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.018550 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:47.018582 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0208 04:10:47.018640 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:47.018690 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.018732 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0208 04:10:47.018788 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.018801 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:47.018863 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0208 04:10:47.018869 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:47.018903 3399 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0208 04:10:47.018919 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:47.019003 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } W0208 04:10:47.019025 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.019038 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0208 04:10:47.019042 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0208 04:10:47.019064 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0208 04:10:47.019098 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0208 04:10:47.019077 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:47.019113 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0208 04:10:47.019132 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0208 04:10:47.019149 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0208 04:10:47.019173 3399 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0208 04:10:47.019121 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0208 04:10:47.019180 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } W0208 04:10:47.019194 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.019196 3399 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0208 04:10:47.019207 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0208 04:10:47.019392 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.27s) === RUN TestLifecyclePrivateSharedIP I0208 04:10:47.019642 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:47.019834 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:47.019870 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:47.019907 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:47.019954 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:47.019990 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:47.020019 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:47.020059 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:47.020084 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:47.020142 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0208 04:10:47.036883 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:47.038832 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.038895 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.038948 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.074794 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.140219 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.141579 3399 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0208 04:10:47.141854 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.142006 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:47.142103 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:47.142053 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.142139 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.142150 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0208 04:10:47.142221 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:47.142241 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:47.142363 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:47.142361 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.142426 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:47.142497 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:47.142526 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:47.142470 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0208 04:10:47.142553 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.143331 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.143500 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.143247 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0208 04:10:47.143703 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0208 04:10:47.144930 3399 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0208 04:10:47.145066 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0208 04:10:47.145114 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0208 04:10:47.145198 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.145350 3399 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0208 04:10:47.145227 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0208 04:10:47.145530 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.145515 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.146143 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.145659 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0208 04:10:47.146249 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0208 04:10:47.146312 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:47.146341 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0208 04:10:47.146303 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0208 04:10:47.146401 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0208 04:10:47.146456 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:47.146833 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.146420 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.146911 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.147000 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:47.147076 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0208 04:10:47.147075 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0208 04:10:47.147238 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:47.147301 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:47.147357 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0208 04:10:47.147466 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0208 04:10:47.147659 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.147612 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0208 04:10:47.147818 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.147983 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" } ] }], VpcId: "vpc-12345678" } I0208 04:10:47.147988 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0208 04:10:47.148115 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I0208 04:10:47.148205 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:47.148271 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:47.148358 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0208 04:10:47.148449 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:47.148479 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:47.148504 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.148530 3399 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] } I0208 04:10:47.148581 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0208 04:10:47.148639 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:47.148665 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:47.148696 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.148722 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:47.148751 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:47.148777 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:47.148800 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0208 04:10:47.148844 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:47.148868 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:47.149492 3399 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0208 04:10:47.149562 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0208 04:10:47.149648 3399 api.go:53] PageSize not implemented I0208 04:10:47.149666 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:47.149696 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } W0208 04:10:47.149766 3399 api.go:53] PageSize not implemented I0208 04:10:47.149724 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.149832 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.149872 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0208 04:10:47.149930 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0208 04:10:47.149914 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.149995 3399 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" }] } I0208 04:10:47.150034 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0208 04:10:47.150054 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0208 04:10:47.150090 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0208 04:10:47.150132 3399 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0208 04:10:47.150165 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0208 04:10:47.150097 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.150182 3399 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0208 04:10:47.150211 3399 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0208 04:10:47.150253 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0208 04:10:47.150375 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0208 04:10:47.150472 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0208 04:10:47.150552 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.150627 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:47.150815 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.150972 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.151028 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.151138 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0208 04:10:47.151263 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:47.151311 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.151432 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:47.151553 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.151642 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.151686 3399 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0208 04:10:47.151733 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.151849 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0208 04:10:47.151926 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0208 04:10:47.152002 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0208 04:10:47.152036 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152124 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152199 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0208 04:10:47.152256 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152327 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152414 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" }] }] } I0208 04:10:47.152491 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" }] }] } I0208 04:10:47.152565 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0208 04:10:47.152631 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0208 04:10:47.152659 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.152721 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.152779 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152882 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.152971 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0208 04:10:47.153060 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.153138 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0208 04:10:47.153163 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0208 04:10:47.153186 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.153344 3399 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0208 04:10:47.153388 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.153433 3399 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.153425 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:47.153515 3399 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0208 04:10:47.153564 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:47.153548 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:47.153589 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.153628 3399 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0208 04:10:47.153676 3399 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0208 04:10:47.153722 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.153822 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0208 04:10:47.153868 3399 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0208 04:10:47.153894 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.153919 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.153950 3399 dns.go:238] Pre-creating DNS records I0208 04:10:47.153964 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.157541 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0208 04:10:47.159376 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.159427 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.159464 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.195854 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.264028 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.265369 3399 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0208 04:10:47.265746 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:47.265756 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0208 04:10:47.265848 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0208 04:10:47.265879 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0208 04:10:47.265918 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.265989 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.266077 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.266211 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.267644 3399 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0208 04:10:47.267764 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.267839 3399 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0208 04:10:47.267879 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0208 04:10:47.268017 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:47.268084 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0208 04:10:47.268069 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0208 04:10:47.268152 3399 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0208 04:10:47.268178 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:47.268205 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0208 04:10:47.268286 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.268366 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.268476 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0208 04:10:47.268502 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.268575 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:47.268600 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0208 04:10:47.268707 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.268752 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0208 04:10:47.268877 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:47.268911 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0208 04:10:47.269008 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.270497 3399 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0208 04:10:47.270610 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } W0208 04:10:47.270658 3399 api.go:53] PageSize not implemented I0208 04:10:47.270659 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.270675 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0208 04:10:47.270701 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.270728 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0208 04:10:47.270751 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0208 04:10:47.270799 3399 api.go:53] PageSize not implemented I0208 04:10:47.270770 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:47.270810 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0208 04:10:47.270853 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0208 04:10:47.270872 3399 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0208 04:10:47.270950 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.271029 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.271129 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:47.271207 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0208 04:10:47.271289 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.271375 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.271556 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.271697 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0208 04:10:47.273192 3399 executor.go:111] Tasks: 107 done / 116 total; 5 can run W0208 04:10:47.273308 3399 api.go:53] PageSize not implemented I0208 04:10:47.273311 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.273332 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph","api-private-shared-ip-exa-ohatqj"] } I0208 04:10:47.273375 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0208 04:10:47.273447 3399 api.go:53] PageSize not implemented I0208 04:10:47.273320 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0208 04:10:47.273458 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0208 04:10:47.273483 3399 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0208 04:10:47.273563 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0208 04:10:47.273602 3399 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0208 04:10:47.273676 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.273765 3399 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0208 04:10:47.275796 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:47.275834 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:47.275856 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:47.275875 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:47.275894 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.275913 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:47.275932 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:47.275950 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:47.275971 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:47.275991 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:47.276016 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0208 04:10:47.276038 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0208 04:10:47.276060 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:47.276080 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:47.276098 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:47.276119 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:47.276144 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0208 04:10:47.276165 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:47.276186 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:47.276206 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0208 04:10:47.276228 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0208 04:10:47.276250 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0208 04:10:47.276271 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.276292 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:47.276313 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0208 04:10:47.276335 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:47.276354 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:47.276375 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:47.276396 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:47.276417 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:47.277293 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:47.277311 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:47.277335 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277374 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0208 04:10:47.277421 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277455 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277482 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277506 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277530 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277557 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0208 04:10:47.277591 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277623 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:47.277642 3399 natgateway.go:117] DescribeNatGateways: { } I0208 04:10:47.277667 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0208 04:10:47.277689 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0208 04:10:47.277718 3399 api.go:53] PageSize not implemented I0208 04:10:47.277728 3399 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0208 04:10:47.277759 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:47.277776 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:47.277787 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:47.277801 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:47.277812 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:47.277841 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.277859 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.277885 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:47.277907 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.277937 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:47.277966 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:47.277988 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0208 04:10:47.278011 3399 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0208 04:10:47.278032 3399 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-config:lt-2 ok autoscaling-group:nodes.private-shared-ip.example.com ok I0208 04:10:47.278387 3399 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0208 04:10:47.278440 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I0208 04:10:47.278443 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0208 04:10:47.278385 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.278482 3399 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } load-balancer:api-private-shared-ip-exa-ohatqj ok autoscaling-group:bastion.private-shared-ip.example.com ok I0208 04:10:47.278494 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.278505 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0208 04:10:47.278443 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0208 04:10:47.278540 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0208 04:10:47.278597 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0208 04:10:47.278623 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.278652 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0208 04:10:47.278531 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W0208 04:10:47.278677 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.278696 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:47.278718 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0208 04:10:47.278748 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0208 04:10:47.278814 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.278823 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-1 ok I0208 04:10:47.278890 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:47.278927 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok I0208 04:10:47.278983 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.279029 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0208 04:10:47.279077 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.279085 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:47.279149 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0208 04:10:47.279162 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0208 04:10:47.279209 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.279199 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0208 04:10:47.279255 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com W0208 04:10:47.279258 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.279267 3399 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0208 04:10:47.279277 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0208 04:10:47.279284 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0208 04:10:47.279308 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0208 04:10:47.279312 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:47.279330 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.279353 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0208 04:10:47.279383 3399 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0208 04:10:47.279404 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0208 04:10:47.279405 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0208 04:10:47.279435 3399 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0208 04:10:47.279465 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0208 04:10:47.279492 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0208 04:10:47.279449 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0208 04:10:47.279545 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.279556 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0208 04:10:47.279612 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:47.279658 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0208 04:10:47.279812 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.26s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0208 04:10:47.279996 3399 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:47.280162 3399 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:47.280190 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:47.280216 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:47.280245 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:47.280269 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:47.280285 3399 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:47.280307 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:47.280322 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:47.280355 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0208 04:10:47.297754 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0208 04:10:47.299970 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.300027 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.336016 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.418197 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.419269 3399 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0208 04:10:47.419401 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.419501 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.419538 3399 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.419569 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.419727 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.419766 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:47.419840 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.419996 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.420050 3399 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.420079 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0208 04:10:47.420191 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:47.420224 3399 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.420300 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:47.420343 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0208 04:10:47.420301 3399 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0208 04:10:47.420364 3399 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0208 04:10:47.420380 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:47.420403 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.420424 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.420526 3399 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.420591 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:47.420626 3399 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0208 04:10:47.420775 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:47.421590 3399 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0208 04:10:47.421723 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0208 04:10:47.421682 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.421755 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0208 04:10:47.421835 3399 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0208 04:10:47.421836 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0208 04:10:47.421912 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.421990 3399 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0208 04:10:47.422012 3399 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.422015 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0208 04:10:47.422105 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:47.422135 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.422160 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.422148 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.422271 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.422442 3399 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.422513 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:47.422547 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.422643 3399 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:47.422681 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:47.422711 3399 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] } I0208 04:10:47.422764 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.422892 3399 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0208 04:10:47.422896 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0208 04:10:47.423033 3399 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0208 04:10:47.423093 3399 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0208 04:10:47.423130 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.423161 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:47.423182 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:47.423213 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.423241 3399 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.424553 3399 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0208 04:10:47.424684 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.424807 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.424868 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.424993 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0208 04:10:47.425026 3399 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0208 04:10:47.425118 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.425298 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.425458 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.425602 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.425767 3399 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.425918 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.426031 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.426171 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.426292 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.426369 3399 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:47.426422 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.426513 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.426602 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.426686 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.426750 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.426860 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.426940 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.427001 3399 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0208 04:10:47.427046 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" }] }] } I0208 04:10:47.427149 3399 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0208 04:10:47.428048 3399 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0208 04:10:47.428198 3399 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0208 04:10:47.428348 3399 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0208 04:10:47.428376 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.428399 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.428433 3399 dns.go:238] Pre-creating DNS records I0208 04:10:47.428453 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.431647 3399 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0208 04:10:47.433395 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.433445 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.470486 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.549809 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.550924 3399 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0208 04:10:47.551093 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.551190 3399 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0208 04:10:47.551295 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.551400 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.551500 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:47.551534 3399 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:47.551560 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.551700 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.551743 3399 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0208 04:10:47.551810 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.553128 3399 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0208 04:10:47.553284 3399 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0208 04:10:47.553227 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.553340 3399 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0208 04:10:47.553365 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.553512 3399 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:47.553559 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0208 04:10:47.553709 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.553758 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.553861 3399 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:47.553925 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0208 04:10:47.554051 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.554093 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.557676 3399 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0208 04:10:47.558010 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.558156 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.558316 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.558514 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.558954 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.559116 3399 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0208 04:10:47.559266 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.560191 3399 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0208 04:10:47.560382 3399 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0208 04:10:47.560522 3399 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0208 04:10:47.562613 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0208 04:10:47.562649 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0208 04:10:47.562669 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0208 04:10:47.562686 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0208 04:10:47.562704 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0208 04:10:47.562725 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0208 04:10:47.562761 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0208 04:10:47.562781 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0208 04:10:47.562799 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0208 04:10:47.562820 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0208 04:10:47.562840 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0208 04:10:47.562862 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0208 04:10:47.562885 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0208 04:10:47.562910 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0208 04:10:47.562930 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0208 04:10:47.562950 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0208 04:10:47.562968 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:47.562990 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0208 04:10:47.563011 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:47.563031 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0208 04:10:47.563050 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0208 04:10:47.563067 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0208 04:10:47.563085 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0208 04:10:47.563105 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0208 04:10:47.563126 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:47.563144 3399 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0208 04:10:47.563945 3399 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0208 04:10:47.563962 3399 keypairs.go:95] DescribeKeyPairs: { } I0208 04:10:47.563985 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564016 3399 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564051 3399 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564086 3399 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564114 3399 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564146 3399 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564170 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564193 3399 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564222 3399 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564255 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0208 04:10:47.564279 3399 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0208 04:10:47.564312 3399 api.go:53] PageSize not implemented I0208 04:10:47.564324 3399 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0208 04:10:47.564341 3399 loadbalancers.go:34] PageSize not implemented I0208 04:10:47.564352 3399 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0208 04:10:47.564366 3399 targetgroups.go:35] PageSize not implemented I0208 04:10:47.564378 3399 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0208 04:10:47.564409 3399 zones.go:93] ListHostedZonesPages { } I0208 04:10:47.564427 3399 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0208 04:10:47.564456 3399 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0208 04:10:47.564480 3399 iamrole.go:89] ListRoles: { } I0208 04:10:47.564510 3399 internetgateways.go:101] DescribeInternetGateways: { } I0208 04:10:47.564534 3399 routetable.go:53] DescribeRouteTables: { } I0208 04:10:47.564555 3399 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0208 04:10:47.564876 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.564935 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0208 04:10:47.564963 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0208 04:10:47.565032 3399 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.565084 3399 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0208 04:10:47.565076 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0208 04:10:47.564929 3399 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.nthsqsresources.longclustername.example.com. ok I0208 04:10:47.565175 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0208 04:10:47.565486 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.565512 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0208 04:10:47.565559 3399 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0208 04:10:47.565654 3399 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0208 04:10:47.565730 3399 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0208 04:10:47.565852 3399 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0208 04:10:47.565875 3399 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0208 04:10:47.565920 3399 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0208 04:10:47.566006 3399 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0208 04:10:47.566065 3399 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0208 04:10:47.566155 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.566164 3399 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0208 04:10:47.566214 3399 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0208 04:10:47.566234 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0208 04:10:47.566274 3399 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0208 04:10:47.566320 3399 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.566360 3399 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0208 04:10:47.566369 3399 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0208 04:10:47.566390 3399 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0208 04:10:47.566438 3399 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0208 04:10:47.566485 3399 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0208 04:10:47.566526 3399 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0208 04:10:47.566693 3399 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.29s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 51.419s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.005s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0208 04:09:56.863972 3605 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.004s === RUN TestPodController I0208 04:09:56.896549 3610 pod.go:59] starting pod controller I0208 04:09:56.896898 3610 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/watchers 0.044s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.005s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5230} route53_test.go:93: Got zone list: [0xc0000a5260] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc0000120c0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5390} route53_test.go:93: Got zone list: [0xc0000a53c0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc0000120d8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0000a54c0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003316b0 0xc0000120f8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5680} route53_test.go:93: Got zone list: [0xc0000a56b0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012118} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5810} route53_test.go:93: Got zone list: [0xc0000a5840] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012140} route53_test.go:187: Successfully added resource record set: {0xc000180600 0xc000012160} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5b20} route53_test.go:93: Got zone list: [0xc0000a5b50] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc0000121f0} route53_test.go:199: Successfully added resource record set: {0xc000180780 0xc000012210} route53_test.go:118: Got 1 recordsets: [0xc0000a5cf0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5e80} route53_test.go:93: Got zone list: [0xc0000a5eb0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc0000122b0} route53_test.go:221: Successfully added resource record set: {0xc000180900 0xc0000122d0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000180900 0xc0000122d0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000368270} route53_test.go:93: Got zone list: [0xc0003682a0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012390} route53_test.go:246: Successfully removed resource set {0xc000180b00 0xc0000123b0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000368580} route53_test.go:93: Got zone list: [0xc0003685b0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012440} route53_test.go:264: Successfully removed resource set {0xc000180c80 0xc000012460} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003688c0} route53_test.go:93: Got zone list: [0xc0003688f0] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc0000124f8} commontests.go:192: Successfully added resource record set: {0xc000180e00 0xc000012518} commontests.go:45: Correctly replaced resource record {0xc000180e00 0xc000012518} -> {0xc000180f00 0xc000012568} commontests.go:142: Got 1 recordsets: [0xc000368c00] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000368e30} route53_test.go:93: Got zone list: [0xc000368e60] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012620} commontests.go:192: Successfully added resource record set: {0xc000181180 0xc000012640} commontests.go:70: Correctly replaced resource record {0xc000181180 0xc000012640} -> {0xc000181280 0xc000012690} commontests.go:142: Got 1 recordsets: [0xc000369170] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003693f0} route53_test.go:93: Got zone list: [0xc000369420] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012758} commontests.go:192: Successfully added resource record set: {0xc000181500 0xc000012778} commontests.go:142: Got 2 recordsets: [0xc000369690 0xc0003696b0] commontests.go:142: Got 2 recordsets: [0xc000369720 0xc000369740] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000369940} route53_test.go:93: Got zone list: [0xc000369970] route53_test.go:98: Got at least 1 zone in list:&{0xc000331470 0xc000012880} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.023s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.011s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450160} clouddns_test.go:71: Got zone list: [0xc00045a5b8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045a558} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450260} clouddns_test.go:71: Got zone list: [0xc00045a738] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045a6d8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450360} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc000458b00 0xc00045a8a0} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450420} clouddns_test.go:71: Got zone list: [0xc00045aa20] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045a9c0} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450540} clouddns_test.go:71: Got zone list: [0xc00045ac90] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045ac30} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450680} clouddns_test.go:71: Got zone list: [0xc00045af48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045aee8} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc0004507e0} clouddns_test.go:71: Got zone list: [0xc00045b368] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045b308} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450940} clouddns_test.go:71: Got zone list: [0xc00045b6b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045b638} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450a80} clouddns_test.go:71: Got zone list: [0xc00045b980] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045b920} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450be0} clouddns_test.go:71: Got zone list: [0xc00045bd10] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc00045bcb0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450d40} clouddns_test.go:71: Got zone list: [0xc000134120] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc0001340f0} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc000450ec0} clouddns_test.go:71: Got zone list: [0xc000134450] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc000134420} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc00049a048 0xc0004510a0} clouddns_test.go:71: Got zone list: [0xc000134990] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000458980 0xc000134960} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.076s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.037s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.049s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0208 04:10:04.577369 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.577656 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.577713 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.577762 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.577833 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.577884 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.577925 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.577964 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.578014 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.591984 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0208 04:10:04.597079 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.597293 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.597333 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.597374 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.597415 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.597449 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.597475 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.597507 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.597531 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.599142 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0208 04:10:04.604085 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.604291 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.604349 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.604391 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.604448 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.604486 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.604525 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.604567 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.604594 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.606190 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.608444 4205 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0208 04:10:04.608471 4205 assetstore.go:111] bin/containerd I0208 04:10:04.608485 4205 assetstore.go:111] bin/containerd-shim I0208 04:10:04.608500 4205 assetstore.go:111] bin/containerd-shim-runc-v1 I0208 04:10:04.608514 4205 assetstore.go:111] bin/containerd-shim-runc-v2 I0208 04:10:04.608527 4205 assetstore.go:111] bin/containerd-stress I0208 04:10:04.608541 4205 assetstore.go:111] bin/ctr I0208 04:10:04.608596 4205 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0208 04:10:04.608617 4205 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0208 04:10:04.613403 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.613614 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.613673 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.613716 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.613761 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.613797 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.613826 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.613860 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.613884 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.615469 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.617606 4205 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.01s) === RUN TestContainerdBuilder_SkipInstall I0208 04:10:04.618986 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.619200 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.619243 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.619299 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.619358 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.619448 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.619478 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.619514 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.619538 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.621105 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.623398 4205 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0208 04:10:04.624392 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.624593 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.624645 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.624687 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.624726 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.624795 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.624837 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.624894 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.624921 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.626762 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.629048 4205 assetstore.go:108] Matching assets for "^docker/": I0208 04:10:04.629073 4205 assetstore.go:111] docker/containerd I0208 04:10:04.629085 4205 assetstore.go:111] docker/containerd-shim I0208 04:10:04.629095 4205 assetstore.go:111] docker/ctr I0208 04:10:04.629104 4205 assetstore.go:111] docker/docker I0208 04:10:04.629115 4205 assetstore.go:111] docker/docker-init I0208 04:10:04.629125 4205 assetstore.go:111] docker/docker-proxy I0208 04:10:04.629136 4205 assetstore.go:111] docker/dockerd I0208 04:10:04.629147 4205 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0208 04:10:04.633633 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.633772 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.633804 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.633836 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.633865 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.633911 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.633925 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.633945 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.633964 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.635047 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.636448 4205 assetstore.go:108] Matching assets for "^docker/": I0208 04:10:04.636468 4205 assetstore.go:111] docker/containerd I0208 04:10:04.636475 4205 assetstore.go:111] docker/containerd-shim I0208 04:10:04.636481 4205 assetstore.go:111] docker/ctr I0208 04:10:04.636487 4205 assetstore.go:111] docker/docker I0208 04:10:04.636493 4205 assetstore.go:111] docker/docker-init I0208 04:10:04.636499 4205 assetstore.go:111] docker/docker-proxy I0208 04:10:04.636507 4205 assetstore.go:111] docker/dockerd I0208 04:10:04.636516 4205 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0208 04:10:04.639135 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.639269 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.639310 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.639336 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.639370 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.639416 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.639432 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.639452 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.639466 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.640617 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.642074 4205 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0208 04:10:04.642364 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.642553 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.642587 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.642663 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.642693 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.642714 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.642731 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.642757 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.642772 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.644036 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:04.645792 4205 assetstore.go:108] Matching assets for "^docker/": I0208 04:10:04.645808 4205 assetstore.go:111] docker/docker I0208 04:10:04.645816 4205 assetstore.go:111] docker/docker-containerd I0208 04:10:04.645822 4205 assetstore.go:111] docker/docker-containerd-ctr I0208 04:10:04.645828 4205 assetstore.go:111] docker/docker-containerd-shim I0208 04:10:04.645834 4205 assetstore.go:111] docker/docker-init I0208 04:10:04.645840 4205 assetstore.go:111] docker/docker-proxy I0208 04:10:04.645845 4205 assetstore.go:111] docker/docker-runc I0208 04:10:04.645850 4205 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0208 04:10:04.649564 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.649718 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.649751 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.649774 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.649803 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.649825 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.649839 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.649858 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.649872 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.652186 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0208 04:10:04.654636 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.654835 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.654896 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.654970 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.655031 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.655084 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.655133 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.655184 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.655209 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.656896 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0208 04:10:04.659045 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.659198 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.659234 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.659276 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.659312 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.659347 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.659373 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.659408 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.659433 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.661212 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.01s) === RUN TestKubeAPIServerBuilder I0208 04:10:04.670967 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.671254 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.671299 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.671340 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.671380 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.671414 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.671438 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.671469 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.671491 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.674133 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0208 04:10:04.687080 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.687360 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.687438 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.687507 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.687579 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.687647 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.687708 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.687774 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.687826 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.690608 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0208 04:10:04.700964 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.701224 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.701281 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.701332 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.701389 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.701436 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.701470 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.701515 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.701546 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.704457 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.03s) === RUN TestAwsIamAuthenticator I0208 04:10:04.730947 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.731197 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.731248 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.731300 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.731348 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.731391 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.731420 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.731457 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.731486 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.735361 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I0208 04:10:04.747098 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:04.747313 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:04.747364 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:04.747407 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:04.747455 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:04.747496 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:04.747528 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:04.747569 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:04.747595 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:04.749647 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.46s) === RUN TestKubeAPIServerBuilderARM64 I0208 04:10:06.205341 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.205552 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.205596 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.205639 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.205690 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.205724 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.205742 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.205767 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.205791 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.207719 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.09s) === RUN TestKubeControllerManagerBuilder I0208 04:10:06.296038 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.296302 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.296352 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.296400 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.296450 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.296490 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.296520 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.296559 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.296585 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.299774 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.02s) === RUN TestKubeControllerManagerBuilderAMD64 I0208 04:10:06.311685 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.311920 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.311971 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.312019 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.312075 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.312121 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.312159 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.312207 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.312239 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.315483 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.07s) === RUN TestKubeControllerManagerBuilderARM64 I0208 04:10:06.383851 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.384041 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.384087 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.384133 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.384182 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.384220 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.384250 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.384288 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.384312 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.386270 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.07s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0208 04:10:06.455657 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.455849 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.455885 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.455917 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.455950 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.455976 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.455993 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.456015 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.456030 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.457821 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0208 04:10:06.461748 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.461905 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.461936 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.461962 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.461993 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.462016 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.462032 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.462054 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.462070 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.463802 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0208 04:10:06.527475 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.527653 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.527690 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.527722 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.527755 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.527782 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.527801 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.527823 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.527838 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.529681 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0208 04:10:06.597945 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.598122 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.598158 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.598187 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.598219 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.598245 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.598261 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.598288 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.598305 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.600132 4205 zones.go:93] ListHostedZonesPages { } W0208 04:10:06.602664 4205 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0208 04:10:06.604405 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.604565 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.604598 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.604625 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.604655 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.604680 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.604701 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.604726 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.604743 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.606551 4205 zones.go:93] ListHostedZonesPages { } W0208 04:10:06.672860 4205 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0208 04:10:06.676328 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.676606 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.676662 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.676717 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.676772 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.676818 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.676856 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.676902 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.676934 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.680305 4205 zones.go:93] ListHostedZonesPages { } W0208 04:10:06.750977 4205 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.08s) === RUN TestKubectlBuilder I0208 04:10:06.754154 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.754342 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.754380 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.754414 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.754454 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.754481 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.754518 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.754564 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.754591 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.756673 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:06.758584 4205 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0208 04:10:06.760579 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.760723 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.760774 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.760802 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.760836 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.760861 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.760879 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.760901 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.760917 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.762221 4205 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.00s) === RUN Test_RunKubeletBuilderWarmPool I0208 04:10:06.765560 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.765708 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.765753 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.765779 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.765807 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.765829 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.765847 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.765867 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.765882 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.767139 4205 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.00s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0208 04:10:06.770308 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.770448 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.770476 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.770505 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.770532 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.770554 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.770571 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.770594 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.770607 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.772273 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:06.773872 4205 assetstore.go:108] Matching assets for "protokube$": I0208 04:10:06.773892 4205 assetstore.go:111] /opt/kops/bin/protokube I0208 04:10:06.773902 4205 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0208 04:10:06.773922 4205 assetstore.go:108] Matching assets for "channels$": I0208 04:10:06.773933 4205 assetstore.go:111] /opt/kops/bin/channels I0208 04:10:06.773941 4205 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.00s) === RUN TestSecretBuilder I0208 04:10:06.775228 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.775370 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.775398 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.775423 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.775450 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.775473 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.775493 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.775515 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.775529 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.779676 4205 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.01s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0208 04:10:06.783308 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.783535 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.783578 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.783621 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.783665 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.783702 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.783730 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.783768 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.783793 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.786427 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:06.789067 4205 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.01s) === RUN TestUpdateServiceBuilderExternal I0208 04:10:06.789808 4205 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:06.790060 4205 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:06.790106 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:06.790146 4205 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:06.790190 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:06.790227 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:06.790257 4205 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:06.790294 4205 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:06.790318 4205 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:06.793013 4205 zones.go:93] ListHostedZonesPages { } I0208 04:10:06.795521 4205 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.01s) PASS ok k8s.io/kops/nodeup/pkg/model 2.308s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.035s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.042s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.033s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.027s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0208 04:10:04.504819 4211 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0208 04:10:04.504943 4211 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.010s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0208 04:10:04.572739 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.572863 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.572912 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.572959 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573008 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0208 04:10:04.573334 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573384 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573425 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573463 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573507 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573540 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573579 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573611 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573654 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573698 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573738 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573782 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573821 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573862 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573898 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573955 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.573999 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.574036 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0208 04:10:04.574073 4212 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0208 04:10:04.574254 4212 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0208 04:10:04.574348 4212 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.092s === RUN TestApplySet I0208 04:10:04.529432 4213 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0208 04:10:04.530536 4213 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0208 04:10:04.531433 4213 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0208 04:10:04.531877 4213 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0208 04:10:04.532826 4213 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0208 04:10:04.533346 4213 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0208 04:10:04.533848 4213 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0208 04:10:04.533906 4213 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0208 04:10:04.534630 4213 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I0208 04:10:04.534715 4213 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I0208 04:10:04.535011 4213 health.go:33] status conditions not found for /v1, Kind=Namespace I0208 04:10:04.535283 4213 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I0208 04:10:04.535344 4213 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I0208 04:10:04.535598 4213 health.go:33] status conditions not found for /v1, Kind=ConfigMap I0208 04:10:04.535861 4213 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I0208 04:10:04.535931 4213 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I0208 04:10:04.536722 4213 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I0208 04:10:04.536767 4213 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.044s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0208 04:10:04.610600 4214 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.09s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0208 04:10:04.611505 4214 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0208 04:10:04.612250 4214 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0208 04:10:04.613110 4214 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0208 04:10:04.613985 4214 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0208 04:10:04.614840 4214 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0208 04:10:04.615656 4214 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0208 04:10:04.616501 4214 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0208 04:10:04.616525 4214 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0208 04:10:04.616534 4214 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0208 04:10:04.616542 4214 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.123s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.007s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.068s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.027s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.091s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.027s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.004s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.004s === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.036s === RUN TestFlagToFalse I0208 04:10:12.083775 5666 featureflag_test.go:32] Created flag Unittest1 I0208 04:10:12.083821 5666 featureflag.go:160] FeatureFlag "UnitTest1"=false I0208 04:10:12.083831 5666 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0208 04:10:12.083896 5666 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.005s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.032s === RUN TestRenderInstanceGroupZones W0208 04:10:12.103445 5668 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.026s === RUN TestWarmPoolOnlyRoll I0208 04:10:24.558824 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.558915 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.558942 6255 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0208 04:10:24.559533 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.559565 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.560745 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.560790 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.565954 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.566009 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.566033 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.566048 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.568936 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.569971 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.570013 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.571171 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.571211 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.576433 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.576515 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.577564 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.577603 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.578771 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.578813 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.584045 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.584097 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.584120 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.584138 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.584991 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.585519 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.585549 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.586677 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.586715 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.591963 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.592017 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.592730 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.592759 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.593901 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.593983 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.599231 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.599302 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.600269 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.600304 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.601468 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.601495 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.606746 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.606784 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.606802 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.606816 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.607779 6255 instancegroups.go:407] Draining the node: "node-2a.local". I0208 04:10:24.608422 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.608454 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.609621 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.609647 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.614864 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.614930 6255 instancegroups.go:407] Draining the node: "node-2b.local". I0208 04:10:24.615849 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.615888 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.617033 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.617054 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.622264 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.622349 6255 instancegroups.go:407] Draining the node: "node-2c.local". I0208 04:10:24.623307 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.623345 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.624478 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.624509 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.629725 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.629774 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0208 04:10:24.630363 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.630405 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.631557 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.631614 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.631659 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.631675 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.631702 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.632830 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.632887 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.632904 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.632933 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.634060 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.634090 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.634138 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.634155 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.634186 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.636473 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.636529 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.636543 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.636569 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.637718 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.637768 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.637787 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.637815 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.638952 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.638980 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.639478 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.639500 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.639528 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.640627 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.640668 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.640687 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.640713 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.641830 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0208 04:10:24.641878 6255 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0208 04:10:24.641896 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.641921 6255 instancegroups.go:443] waiting for 1ms after terminating instance W0208 04:10:24.643057 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0208 04:10:24.643088 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0208 04:10:24.643446 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.643477 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.644603 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.644640 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.644661 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.644707 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.644732 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.644754 6255 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.644768 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.645553 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.646325 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.646353 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.647558 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.647613 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.647642 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.647682 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.648610 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.648648 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.649773 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.649808 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.649828 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.649850 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.649872 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.649889 6255 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.649904 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.650859 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.651637 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.651671 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.652796 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.652830 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.652852 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.652890 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.653736 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.653768 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.654899 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.654931 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.654953 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.655003 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.655894 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.655926 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.657048 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.657079 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.657098 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.657114 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.657131 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.657146 6255 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.657161 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.658152 6255 instancegroups.go:407] Draining the node: "node-2a.local". I0208 04:10:24.658922 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.658959 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.660099 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.660134 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.660154 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.660192 6255 instancegroups.go:407] Draining the node: "node-2b.local". I0208 04:10:24.661006 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.661040 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.662165 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.662198 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.662220 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.662265 6255 instancegroups.go:407] Draining the node: "node-2c.local". I0208 04:10:24.663032 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.663063 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.664951 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.664981 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0208 04:10:24.665001 6255 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0208 04:10:24.665025 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0208 04:10:24.665253 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0208 04:10:24.665452 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.665478 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.666594 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.666611 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.671797 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.671857 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.671872 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.671881 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.672430 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.673009 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.673035 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.674153 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.674179 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.679387 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.679452 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.680324 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.680358 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.681496 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.681526 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.686724 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.686781 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.686806 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.686826 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.687983 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.688909 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.688947 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.690042 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.690070 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.695258 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.695305 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.696001 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.696030 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.697155 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.697181 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.702384 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.702443 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.703410 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.703446 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.704575 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.704603 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.709835 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.709874 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.709899 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.709914 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.711050 6255 instancegroups.go:407] Draining the node: "node-2a.local". I0208 04:10:24.711877 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.711912 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.713027 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.713046 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.718232 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.718300 6255 instancegroups.go:407] Draining the node: "node-2b.local". I0208 04:10:24.719313 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.719349 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.720469 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.720491 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.725697 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.725773 6255 instancegroups.go:407] Draining the node: "node-2c.local". I0208 04:10:24.726785 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.726826 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.727950 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.727979 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.733171 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.733213 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0208 04:10:24.733397 6255 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0208 04:10:24.733807 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.733835 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.734946 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.734996 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.735021 6255 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0208 04:10:24.735415 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.735445 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.736570 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.736595 6255 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0208 04:10:24.736614 6255 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0208 04:10:24.736802 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.736828 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.736844 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.736860 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0208 04:10:24.736878 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0208 04:10:24.737026 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.737047 6255 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0208 04:10:24.737063 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.737079 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0208 04:10:24.737095 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0208 04:10:24.737313 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.737338 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.738452 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.738482 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.743688 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.743754 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.743782 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.743800 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.744705 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.745322 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.745349 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.746463 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.746498 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.746521 6255 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0208 04:10:24.746872 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.746902 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.748042 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.748065 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.753272 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.753343 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.753372 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.753388 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.754293 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.755224 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.755269 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.756452 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.756511 6255 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0208 04:10:24.756545 6255 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0208 04:10:24.757006 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.757034 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.757051 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.758274 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.759205 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.759247 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.760386 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.760422 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.765678 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.765745 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.766505 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.766534 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.767658 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.767701 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.772887 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.772929 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.773579 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.773603 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.774714 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.774740 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.779948 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.779999 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0208 04:10:24.780373 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.780399 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.780424 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.780443 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0208 04:10:24.780464 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0208 04:10:24.780732 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.780758 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.780772 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.780798 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0208 04:10:24.780817 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0208 04:10:24.781038 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.781063 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.781078 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.781092 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0208 04:10:24.781108 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0208 04:10:24.781320 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.781342 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.781354 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.782419 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.783174 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.783204 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.784424 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.784466 6255 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0208 04:10:24.784485 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.784506 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0208 04:10:24.784523 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0208 04:10:24.784750 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.784781 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.784796 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.786395 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.787042 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.787075 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.788224 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.788253 6255 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0208 04:10:24.788273 6255 instancegroups.go:482] Cluster did not validate within 0s E0208 04:10:24.788287 6255 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0208 04:10:24.788297 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0208 04:10:24.788498 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.788518 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.789647 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.789694 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.790832 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.796059 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.797239 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.803213 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.804364 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.809598 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.809702 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.809736 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.809757 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.810633 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.811421 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.811460 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.812609 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.812640 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.817843 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.817902 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.818526 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.818552 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.819724 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.819764 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.824963 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.825015 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.825035 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.825046 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.825689 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.826143 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.826167 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.827308 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.827336 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.832583 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.832651 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.833250 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.833288 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.834405 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.834445 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.839675 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.839758 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.840747 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.840783 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.841940 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.841976 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.847207 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.847257 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.847289 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.847309 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.848387 6255 instancegroups.go:407] Draining the node: "node-2a.local". I0208 04:10:24.849157 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.849194 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.850355 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.850390 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.855627 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.855717 6255 instancegroups.go:407] Draining the node: "node-2b.local". I0208 04:10:24.856640 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.856676 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.857824 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.857888 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.863377 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.863469 6255 instancegroups.go:407] Draining the node: "node-2c.local". I0208 04:10:24.864423 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.864462 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.865624 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.865680 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.870853 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.870904 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0208 04:10:24.871287 6255 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0208 04:10:24.871323 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.872494 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.872555 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.874527 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.875624 6255 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0208 04:10:24.876738 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.881928 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.882003 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.882039 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.882058 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.882664 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.883255 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.883288 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.884436 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.884455 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.890649 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.890741 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.891497 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.891526 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.892716 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.892752 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.897942 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.897985 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.897999 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.898017 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.898653 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.899133 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.899159 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.900321 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.900358 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.905533 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.905590 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.906231 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.906258 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.907420 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.907459 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.912646 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.912723 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:24.913433 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:24.913462 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.914614 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.914663 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.919859 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.919887 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.919909 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.919923 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.923321 6255 instancegroups.go:407] Draining the node: "node-2a.local". I0208 04:10:24.924136 6255 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0208 04:10:24.924173 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.925335 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.925359 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.930547 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.930596 6255 instancegroups.go:407] Draining the node: "node-2b.local". I0208 04:10:24.931177 6255 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0208 04:10:24.931208 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.932362 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.932388 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.938078 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.938160 6255 instancegroups.go:407] Draining the node: "node-2c.local". I0208 04:10:24.938776 6255 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0208 04:10:24.938803 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.939948 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.939970 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.945151 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.945187 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0208 04:10:24.945615 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.945631 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.945641 6255 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0208 04:10:24.946199 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:24.946686 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:24.946709 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.947859 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.947885 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.953087 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.953134 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:24.953719 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:24.953745 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.954882 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.954903 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.960256 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.960285 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.01s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0208 04:10:24.960522 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.960539 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.960547 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.961151 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:24.961746 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:24.961771 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.962871 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.962894 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.968081 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.968112 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:24.968606 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:24.968629 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:24.969767 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.969787 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.974962 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.974984 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.01s) === RUN TestRollingUpdateDisabled I0208 04:10:24.975208 6255 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W0208 04:10:24.975232 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0208 04:10:24.975244 6255 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W0208 04:10:24.975253 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0208 04:10:24.975264 6255 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W0208 04:10:24.975269 6255 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0208 04:10:24.975280 6255 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0208 04:10:24.975292 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0208 04:10:24.975467 6255 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I0208 04:10:24.975501 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:24.976637 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.976657 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.981850 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.981875 6255 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I0208 04:10:24.981908 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.981921 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.981934 6255 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0208 04:10:24.982459 6255 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I0208 04:10:24.982477 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.982488 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.982495 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:24.982965 6255 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0208 04:10:24.983000 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:24.984108 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.984128 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.990545 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.990667 6255 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I0208 04:10:24.990725 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.990777 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.990814 6255 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0208 04:10:24.991712 6255 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0208 04:10:24.991758 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:24.992882 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.992910 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:24.998067 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.998102 6255 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0208 04:10:24.998118 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0208 04:10:24.998393 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:24.998416 6255 instancegroups.go:510] Cluster validated. I0208 04:10:24.998426 6255 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0208 04:10:24.999870 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:25.000401 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:25.000434 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.001542 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.001570 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.001613 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:25.001682 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:25.002446 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:25.002663 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:25.002694 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.003828 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.003867 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.104197 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.104256 6255 instancegroups.go:407] Draining the node: "node-1e.local". I0208 04:10:25.104327 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:25.104921 6255 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0208 04:10:25.105296 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:25.105325 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.106448 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.106486 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.106521 6255 instancegroups.go:407] Draining the node: "node-1f.local". I0208 04:10:25.107293 6255 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0208 04:10:25.107332 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.108479 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.108520 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.208923 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.208982 6255 instancegroups.go:407] Draining the node: "node-1g.local". I0208 04:10:25.209586 6255 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0208 04:10:25.209621 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.210771 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.210799 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.210811 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0208 04:10:25.210996 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.211020 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.211034 6255 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0208 04:10:25.212273 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:25.212452 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:25.212969 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:25.213320 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:25.213346 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.214490 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.214534 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.315014 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.315083 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:25.315089 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:25.315802 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:25.316203 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:25.316232 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.317398 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.317457 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.317488 6255 instancegroups.go:407] Draining the node: "node-1e.local". I0208 04:10:25.318069 6255 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0208 04:10:25.318099 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.319250 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.319303 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.419722 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.419798 6255 instancegroups.go:407] Draining the node: "node-1f.local". I0208 04:10:25.420540 6255 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0208 04:10:25.420581 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.421696 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.421721 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.421733 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0208 04:10:25.421902 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.421922 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.421930 6255 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I0208 04:10:25.423156 6255 instancegroups.go:407] Draining the node: "master-1a.local". I0208 04:10:25.423571 6255 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0208 04:10:25.423600 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.424707 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.424731 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.424757 6255 instancegroups.go:407] Draining the node: "master-1c.local". I0208 04:10:25.424825 6255 instancegroups.go:407] Draining the node: "master-1b.local". I0208 04:10:25.425495 6255 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0208 04:10:25.425645 6255 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0208 04:10:25.425682 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.426802 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.426847 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.527240 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.527339 6255 instancegroups.go:407] Draining the node: "master-1e.local". I0208 04:10:25.527361 6255 instancegroups.go:407] Draining the node: "master-1d.local". I0208 04:10:25.528466 6255 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0208 04:10:25.528718 6255 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0208 04:10:25.528764 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.529917 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.529957 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.530000 6255 instancegroups.go:407] Draining the node: "master-1f.local". I0208 04:10:25.530739 6255 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0208 04:10:25.530774 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.531905 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.531969 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.632374 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.632449 6255 instancegroups.go:407] Draining the node: "master-1g.local". I0208 04:10:25.633037 6255 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0208 04:10:25.633071 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.634191 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.634229 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.634252 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0208 04:10:25.634521 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.634556 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.634572 6255 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0208 04:10:25.636443 6255 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0208 04:10:25.636507 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:25.637651 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.637678 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.637687 6255 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0208 04:10:25.637719 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:25.638821 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.638852 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.638890 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:25.638940 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:25.639951 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:25.640252 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:25.640301 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.641461 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.641514 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.741969 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.742039 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:25.742099 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:25.742762 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:25.743149 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:25.743177 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.744365 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.744401 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.744426 6255 instancegroups.go:407] Draining the node: "node-1e.local". I0208 04:10:25.744913 6255 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0208 04:10:25.744948 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.746092 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.746138 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.846601 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.846673 6255 instancegroups.go:407] Draining the node: "node-1f.local". I0208 04:10:25.847508 6255 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0208 04:10:25.847563 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.857659 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.857721 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.857742 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0208 04:10:25.858021 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.858051 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.858065 6255 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0208 04:10:25.859672 6255 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0208 04:10:25.859722 6255 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0208 04:10:25.859760 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:25.860937 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.860987 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.861039 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:25.861103 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:25.862035 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:25.862201 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:25.862228 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.863346 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.863385 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.963798 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.963855 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:25.963940 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:25.964519 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:25.964870 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:25.964898 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.966034 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.966061 6255 instancegroups.go:510] Cluster validated. I0208 04:10:25.966082 6255 instancegroups.go:407] Draining the node: "node-1e.local". I0208 04:10:25.966535 6255 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0208 04:10:25.966574 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:25.967711 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:25.967747 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.068257 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.068315 6255 instancegroups.go:407] Draining the node: "node-1f.local". I0208 04:10:26.068906 6255 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0208 04:10:26.068947 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.070063 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.070092 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.070104 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0208 04:10:26.070268 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.070283 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.070291 6255 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0208 04:10:26.070720 6255 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0208 04:10:26.070741 6255 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0208 04:10:26.070762 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:26.071889 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.071909 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.077098 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.077154 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:26.077175 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:26.078152 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:26.078189 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.078370 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:26.078397 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.079327 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.079348 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.084577 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.084607 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0208 04:10:26.084823 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.084840 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.084848 6255 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0208 04:10:26.085349 6255 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0208 04:10:26.085384 6255 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0208 04:10:26.085391 6255 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0208 04:10:26.085413 6255 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0208 04:10:26.085431 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:26.086567 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.086588 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.091761 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.091804 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:26.091837 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:26.092660 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:26.092688 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.092884 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:26.092907 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.093831 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.093852 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.099086 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.099109 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0208 04:10:26.099302 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.099324 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.099332 6255 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I0208 04:10:26.100039 6255 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0208 04:10:26.100078 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:26.101213 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.101239 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.101247 6255 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0208 04:10:26.101277 6255 instancegroups.go:179] waiting for 1ms after detaching instance I0208 04:10:26.102384 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.102418 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.102446 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:26.102479 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:26.102565 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:26.103690 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:26.103783 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.104129 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:26.104170 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.104319 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:26.104343 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.105085 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.205610 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.205684 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:26.206300 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:26.206336 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.207488 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.207548 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.207574 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0208 04:10:26.207870 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.207905 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.207923 6255 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0208 04:10:26.210206 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:26.210986 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:26.211028 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.212193 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.212227 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.212255 6255 instancegroups.go:407] Draining the node: "node-1e.local". I0208 04:10:26.212301 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:26.212839 6255 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0208 04:10:26.213212 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:26.213238 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.214356 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.214422 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.314768 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.314827 6255 instancegroups.go:407] Draining the node: "node-1g.local". I0208 04:10:26.314870 6255 instancegroups.go:407] Draining the node: "node-1f.local". I0208 04:10:26.315810 6255 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0208 04:10:26.316195 6255 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0208 04:10:26.316252 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.317408 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.317436 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.317459 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:26.317924 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:26.317950 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.319039 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.319099 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.419502 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.419560 6255 instancegroups.go:407] Draining the node: "node-1d.local". I0208 04:10:26.420173 6255 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0208 04:10:26.420208 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.421353 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.421382 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.421395 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0208 04:10:26.421563 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.421579 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.421588 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0208 04:10:26.421599 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0208 04:10:26.421608 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0208 04:10:26.421621 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0208 04:10:26.421723 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.421737 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.421745 6255 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0208 04:10:26.422346 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0208 04:10:26.422363 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0208 04:10:26.422373 6255 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0208 04:10:26.422399 6255 instancegroups.go:407] Draining the node: "node-1a.local". I0208 04:10:26.422835 6255 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0208 04:10:26.422855 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.423965 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.423985 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.429367 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.429410 6255 instancegroups.go:407] Draining the node: "node-1b.local". I0208 04:10:26.431325 6255 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0208 04:10:26.431365 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.432527 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.432559 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.437763 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.437803 6255 instancegroups.go:407] Draining the node: "node-1c.local". I0208 04:10:26.438300 6255 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0208 04:10:26.438324 6255 instancegroups.go:443] waiting for 1ms after terminating instance I0208 04:10:26.439402 6255 instancegroups.go:477] Validating the cluster. I0208 04:10:26.439423 6255 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0208 04:10:26.444680 6255 instancegroups.go:510] Cluster validated. I0208 04:10:26.444726 6255 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 2.006s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.007s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.017s === RUN TestParse E0208 04:10:24.454868 6261 versions.go:96] unable to parse Kubernetes version "" E0208 04:10:24.454969 6261 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.010s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.041s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.16s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.254s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.109s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.096s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.110s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0208 04:10:24.513535 6266 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.073s === RUN Test_RunEtcdManagerBuilder I0208 04:10:24.539767 6258 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0208 04:10:24.590531 6258 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W0208 04:10:24.592288 6258 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.03s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.158s === RUN Test_RunKubeApiserverBuilder I0208 04:10:24.527053 6267 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.105s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.094s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.037s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.103s === RUN TestServerGroupBuilder I0208 04:10:33.003333 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.017462 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.030762 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.051599 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.108649 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.133777 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.157513 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.175813 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.188009 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.250600 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.259560 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.272276 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.280067 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.286155 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.293643 7082 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.30s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.383s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.032s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.013s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.031s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.79s) --- PASS: TestIssueCert/ca (1.40s) --- PASS: TestIssueCert/client (0.22s) --- PASS: TestIssueCert/clientOneYear (0.17s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.820s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.005s === RUN TestAddUntaggedRouteTables I0208 04:10:32.998497 7084 routetable.go:53] DescribeRouteTables: { } W0208 04:10:32.998655 7084 tags.go:67] cluster tag not found on route-table:rtb-1234 I0208 04:10:32.998676 7084 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0208 04:10:32.998837 7084 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0208 04:10:32.998919 7084 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0208 04:10:32.999051 7084 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0208 04:10:32.999191 7084 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0208 04:10:32.999292 7084 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0208 04:10:32.999409 7084 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0208 04:10:32.999541 7084 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0208 04:10:32.999587 7084 volumes.go:128] DescribeVolumes: { } W0208 04:10:32.999640 7084 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0208 04:10:32.999847 7084 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:33.000104 7084 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:33.000168 7084 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:33.000263 7084 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:33.000346 7084 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:33.000410 7084 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:33.000440 7084 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:33.000498 7084 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:33.000534 7084 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:33.000565 7084 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0208 04:10:33.000610 7084 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0208 04:10:33.000645 7084 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0208 04:10:33.000696 7084 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0208 04:10:33.000734 7084 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0208 04:10:33.000782 7084 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0208 04:10:33.000833 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0208 04:10:33.000871 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0208 04:10:33.000907 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0208 04:10:33.000942 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0208 04:10:33.000983 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0208 04:10:33.001020 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0208 04:10:33.001065 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0208 04:10:33.001407 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0208 04:10:33.001446 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0208 04:10:33.001478 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0208 04:10:33.001522 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0208 04:10:33.001556 7084 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.079s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.034s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.067s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.005s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.006s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.004s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.003s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.004s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.040s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.071s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0208 04:10:41.572155 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572244 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572257 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572270 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572281 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572292 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572305 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572317 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572348 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572365 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572378 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572391 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572403 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572414 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.572425 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.572437 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.573850 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.573867 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.573928 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.573939 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.573994 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574005 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574059 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574070 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574124 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574135 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574191 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574203 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574278 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574290 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574358 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574370 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574420 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574431 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574488 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574499 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574573 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574589 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574641 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574652 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574712 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574724 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574778 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574789 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574840 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574851 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.574932 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.574945 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575000 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575011 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575064 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575075 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575135 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575146 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575237 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575249 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575304 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575316 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575377 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575389 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575438 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575448 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575505 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575516 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575569 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575588 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575644 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575655 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575732 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575746 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575798 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575810 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575868 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575880 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.575929 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.575940 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576011 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576023 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576087 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576098 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576165 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576185 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576268 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576291 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576378 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576397 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576493 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576511 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576611 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576632 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576712 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576739 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576842 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576861 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.576969 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.576990 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577073 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577092 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577215 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577235 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577337 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577358 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577447 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577469 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577558 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577580 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577670 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577692 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577778 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577799 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.577906 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.577938 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578036 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578058 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578145 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578166 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578254 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578276 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578362 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578382 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578484 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578504 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578590 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578610 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578715 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578738 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578840 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578863 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.578953 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.578975 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579094 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579118 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579217 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579239 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579320 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579338 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579417 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579433 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579500 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579515 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579586 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579602 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579729 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579744 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579807 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579823 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579901 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.579916 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.579998 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580014 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580080 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580095 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580170 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580186 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580248 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580263 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580332 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580347 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580419 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580435 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580501 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580517 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580595 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580610 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580676 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580691 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580758 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580774 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580845 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580860 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.580922 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.580937 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581017 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581033 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581105 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581120 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581190 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581206 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581278 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581293 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581365 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581379 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581452 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581472 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581573 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581588 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581652 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581667 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581735 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581750 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581825 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581840 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581903 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.581923 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.581995 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582010 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582081 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582097 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582172 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582187 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582253 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582269 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582354 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582379 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582468 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582484 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582558 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582574 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582640 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582656 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582726 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582742 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.582804 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.582820 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.01s) === RUN TestRecoversFromBadNesting W0208 04:10:41.583484 8545 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0208 04:10:41.583499 8545 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0208 04:10:41.583512 8545 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0208 04:10:41.583522 8545 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0208 04:10:41.583534 8545 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0208 04:10:41.583544 8545 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0208 04:10:41.583553 8545 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0208 04:10:41.583591 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583607 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583619 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.583630 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583642 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.583653 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583665 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.583676 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583688 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.583704 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583722 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583734 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583745 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583757 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.583768 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.583780 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598262 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598287 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598352 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598365 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598435 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598446 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598504 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598515 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598564 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598575 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598624 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598641 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598728 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598740 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598793 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598804 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598853 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598863 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.598919 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.598930 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599006 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599024 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599099 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599111 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599177 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599189 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599240 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599251 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599301 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599316 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599372 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599393 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599452 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599464 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599522 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599533 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599586 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599597 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599657 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599669 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599727 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599738 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599790 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599804 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599853 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599864 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599927 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599938 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.599986 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.599997 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600050 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600061 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600115 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600127 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600185 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600201 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600261 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600274 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600331 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600342 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600409 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600421 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600476 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600487 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600538 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600549 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600604 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600615 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600663 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600674 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600730 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600741 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600817 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600829 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600881 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600898 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.600958 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.600969 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601022 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601033 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601089 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601101 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601156 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601167 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601235 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601247 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601306 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601317 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601373 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601383 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601431 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601442 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601494 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601505 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601560 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601573 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601634 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601646 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601701 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601712 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601759 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601770 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601829 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601843 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601892 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601903 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.601957 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.601974 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602044 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602056 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602114 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602125 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602172 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602189 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602241 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602252 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602306 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602317 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602394 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602407 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602457 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602468 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602515 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602526 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602584 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602595 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602653 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602665 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602727 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602738 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602787 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602798 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602860 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602872 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602929 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.602940 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.602991 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603002 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603049 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603065 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603116 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603127 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603178 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603188 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603236 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603246 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603297 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603308 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603359 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603370 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603439 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603450 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603504 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603515 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603562 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603573 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603638 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603666 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603722 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603734 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603786 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603798 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603853 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603864 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603918 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.603930 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.603988 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604000 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604056 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604068 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604116 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604131 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604179 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604189 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604247 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604258 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604311 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604327 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604380 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604391 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604445 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604456 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604526 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604547 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604596 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604607 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604661 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604672 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604723 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604734 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604788 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604799 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604850 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604861 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604908 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604919 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0208 04:10:41.604966 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0208 04:10:41.604976 8545 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.02s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.038s === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.075s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.01s) PASS ok k8s.io/kops/tests/codecs 0.035s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.062s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.04s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.01s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.117s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0208 04:10:43.776091 9002 files.go:57] Writing file "/tmp/TestWriteFile4274701448/001/SubDir/test1.tmp" I0208 04:10:43.776308 9002 files.go:113] Changing file mode for "/tmp/TestWriteFile4274701448/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.01s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.045s === RUN TestBootstrapChannelBuilder_BuildTasks I0208 04:10:43.951959 9105 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:43.952214 9105 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:43.952260 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:43.952298 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:43.952366 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:43.952416 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:43.952437 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:43.952458 9105 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:43.952486 9105 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:43.965068 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.043203 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.157117 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.214133 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.303707 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.404471 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.473423 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.563814 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:44.681963 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.202910 9105 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.31s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0208 04:10:45.260927 9105 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:45.261123 9105 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:45.261195 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:45.261243 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:45.261292 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:45.261330 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:45.261359 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:45.261400 9105 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:45.261425 9105 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:45.261456 9105 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0208 04:10:45.262674 9105 zones.go:93] ListHostedZonesPages { } I0208 04:10:45.327435 9105 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.07s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0208 04:10:45.327622 9105 featureflag.go:160] FeatureFlag "ImageDigest"=false I0208 04:10:45.327764 9105 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:45.327817 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0208 04:10:45.327851 9105 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0208 04:10:45.327881 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0208 04:10:45.327907 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0208 04:10:45.327923 9105 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0208 04:10:45.327945 9105 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0208 04:10:45.327962 9105 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0208 04:10:45.329173 9105 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.13s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.19s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.11s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0208 04:10:45.778581 9105 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0208 04:10:45.814405 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814431 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814442 9105 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0208 04:10:45.814462 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0208 04:10:45.814473 9105 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0208 04:10:45.814492 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0208 04:10:45.814510 9105 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0208 04:10:45.814529 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814541 9105 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0208 04:10:45.814557 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814568 9105 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0208 04:10:45.814586 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814611 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814631 9105 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0208 04:10:45.814641 9105 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0208 04:10:45.814651 9105 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0208 04:10:45.814661 9105 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0208 04:10:45.814670 9105 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0208 04:10:45.814682 9105 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0208 04:10:45.814692 9105 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0208 04:10:45.814702 9105 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0208 04:10:45.815813 9105 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.962s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0208 04:10:43.935243 9106 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3801490618/001 I0208 04:10:43.937140 9106 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3801490618/001 --- PASS: TestAutoscalingGroupTerraformRender (0.00s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0208 04:10:43.945575 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0208 04:10:43.945711 9106 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0208 04:10:43.945845 9106 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0208 04:10:43.946179 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.946385 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.946489 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.946533 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.946801 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.946866 9106 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0208 04:10:43.946928 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.947032 9106 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0208 04:10:43.947157 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0208 04:10:43.947355 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.947426 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.947498 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.947531 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.947592 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.947666 9106 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0208 04:10:43.947763 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0208 04:10:43.948217 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.948319 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.948484 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.948551 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.948612 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.948657 9106 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0208 04:10:43.948836 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.949137 9106 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0208 04:10:43.949224 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.949282 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.949328 9106 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0208 04:10:43.949431 9106 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.949667 9106 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0208 04:10:43.949754 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.949798 9106 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" },{ Key: "AssociatedElasticIp", Value: "192.0.2.1" }] } I0208 04:10:43.949862 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0208 04:10:43.950070 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.950141 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.950226 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.950260 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.950307 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.950349 9106 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0208 04:10:43.950414 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.950511 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.950557 9106 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0208 04:10:43.950621 9106 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.950709 9106 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0208 04:10:43.950756 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0208 04:10:43.950823 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0208 04:10:43.951053 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0208 04:10:43.951119 9106 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0208 04:10:43.951162 9106 internetgateways.go:66] CreateInternetGateway: { } I0208 04:10:43.951191 9106 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0208 04:10:43.951378 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.951447 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.951517 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.951548 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.951667 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.951715 9106 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0208 04:10:43.951770 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.951861 9106 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0208 04:10:43.951978 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0208 04:10:43.952281 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.952359 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.952431 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.952459 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.952517 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.952582 9106 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0208 04:10:43.952676 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0208 04:10:43.955685 9106 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1029878465/001 I0208 04:10:43.958398 9106 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1029878465/001 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0208 04:10:43.959347 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.959429 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.959594 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.959665 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.959706 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.959753 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.959858 9106 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0208 04:10:43.960170 9106 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0208 04:10:43.960249 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0208 04:10:43.960315 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0208 04:10:43.960514 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.960588 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.960665 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.960696 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.960744 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.960786 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.960851 9106 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0208 04:10:43.960971 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0208 04:10:43.961582 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.961655 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.961799 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.961860 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.961903 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.961947 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.962015 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.962198 9106 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0208 04:10:43.962273 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.962316 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.962369 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0208 04:10:43.962570 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.962640 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.962714 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.962745 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.962790 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.962835 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.962898 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.962990 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.963034 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0208 04:10:43.963458 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.963528 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.963676 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.963746 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.963788 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.963831 9106 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0208 04:10:43.963929 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.964098 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.964310 9106 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:43.964309 9106 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0208 04:10:43.964393 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.964425 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.964455 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0208 04:10:43.964731 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.964781 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.964829 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.964845 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.964871 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.964895 9106 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0208 04:10:43.964959 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.965010 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.965031 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.965080 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0208 04:10:43.965417 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.965465 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.965564 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.965608 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.965646 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.965690 9106 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0208 04:10:43.965728 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.965775 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.965945 9106 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0208 04:10:43.965943 9106 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0208 04:10:43.966008 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.966043 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.966077 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0208 04:10:43.966249 9106 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0208 04:10:43.966300 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.966344 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.966359 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.966383 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.966406 9106 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0208 04:10:43.966439 9106 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0208 04:10:43.966536 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.966569 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.966661 9106 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0208 04:10:43.966903 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0208 04:10:43.967019 9106 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0208 04:10:43.967237 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.967307 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.967385 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.967414 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.967505 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.967534 9106 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0208 04:10:43.967567 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.967602 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.967717 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0208 04:10:43.967742 9106 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0208 04:10:43.967775 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0208 04:10:43.967956 9106 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0208 04:10:43.967998 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.968036 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.968050 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.968079 9106 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0208 04:10:43.968112 9106 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0208 04:10:43.968172 9106 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0208 04:10:43.968417 9106 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:43.968482 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.968575 9106 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0208 04:10:43.968617 9106 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0208 04:10:43.968638 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.968660 9106 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:43.968768 9106 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:43.968819 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0208 04:10:43.968861 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.968874 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.968895 9106 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0208 04:10:43.968917 9106 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0208 04:10:43.969130 9106 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0208 04:10:43.969266 9106 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:43.969308 9106 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0208 04:10:43.969350 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0208 04:10:43.969363 9106 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0208 04:10:43.969401 9106 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.099s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.055s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.030s === RUN TestDiskRenderAzure I0208 04:10:44.347767 9308 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0208 04:10:44.348413 9308 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0208 04:10:44.348889 9308 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0208 04:10:44.349716 9308 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0208 04:10:44.350190 9308 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0208 04:10:44.350674 9308 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0208 04:10:44.351159 9308 resourcegroup.go:103] Creating a new Resource Group with name: rg I0208 04:10:44.351180 9308 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0208 04:10:44.351491 9308 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0208 04:10:44.352572 9308 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0208 04:10:44.353038 9308 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0208 04:10:44.353471 9308 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0208 04:10:44.354071 9308 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0208 04:10:44.354497 9308 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.035s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.038s === RUN TestProjectIAMBinding I0208 04:10:47.504057 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.504657 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.504806 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.505301 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.505460 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.505629 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0208 04:10:47.505892 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.505991 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.506117 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.506246 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.506375 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.506431 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0208 04:10:47.506631 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.506919 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.507044 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.507382 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0208 04:10:47.507517 10268 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0208 04:10:47.507645 10268 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.062s === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.040s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.034s === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.029s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.006s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.028s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0208 04:10:50.609255 10530 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0208 04:10:50.609342 10530 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0208 04:10:50.609359 10530 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0208 04:10:50.609379 10530 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0208 04:10:50.609463 10530 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0208 04:10:50.609490 10530 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0208 04:10:50.609518 10530 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0208 04:10:50.610388 10530 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.054s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.005s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.005s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.006s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.006s === RUN TestGetProxyEnvVars W0208 04:10:50.579952 10536 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.020s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.009s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.006s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.005s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.022s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0208 04:10:53.725901 10619 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.053s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 134.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r2.apk >>> kops: Build complete at Wed, 08 Feb 2023 04:10:57 +0000 elapsed time 0h 3m 20s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20230208.040739) (2/3) Purging kubernetes (1.26.1-r1) (3/3) Purging go (1.20-r0) Executing busybox-1.36.0-r3.trigger OK: 392 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...