>>> kops: Building testing/kops 1.25.3-r4 (using abuild 3.10.0-r0) started Thu, 06 Apr 2023 18:54:19 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.26.1-r3) (2/3) Installing go (1.20.3-r0) (3/3) Installing .makedepends-kops (20230406.185420) Executing busybox-1.36.0-r5.trigger OK: 565 MiB in 104 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 39 31.1M 39 12.2M 0 0 12.0M 0 0:00:02 0:00:01 0:00:01 12.0M 100 31.1M 100 31.1M 0 0 23.0M 0 0:00:01 0:00:01 --:--:-- 23.0M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading github.com/google/uuid v1.3.0 go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading k8s.io/apimachinery v0.25.4 go: downloading google.golang.org/api v0.103.0 go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading k8s.io/klog v1.0.0 go: downloading golang.org/x/crypto v0.3.0 go: downloading k8s.io/api v0.25.4 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/client-go v0.25.4 go: downloading github.com/spf13/viper v1.14.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/stretchr/testify v1.8.1 go: downloading github.com/go-logr/logr v1.2.3 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/kubectl v0.25.4 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/pkg/sftp v1.13.5 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/google/go-cmp v0.5.9 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading k8s.io/component-base v0.25.4 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading golang.org/x/sys v0.2.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/net v0.2.0 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading golang.org/x/term v0.2.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading golang.org/x/sync v0.1.0 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/golang/protobuf v1.5.2 go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading golang.org/x/text v0.4.0 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading google.golang.org/grpc v1.50.1 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.4 go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading golang.org/x/tools v0.1.8 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading go.opencensus.io v0.24.0 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/moby/locker v1.0.1 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0406 18:56:35.585456 36653 addon.go:233] addon test wants to update all nodes I0406 18:56:35.587859 36653 addon.go:233] addon test wants to update worker nodes I0406 18:56:35.588410 36653 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0406 18:56:35.589215 36653 addon.go:266] installing PKI for "test" I0406 18:56:35.922255 36653 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.28s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.359s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.077s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] === RUN TestCreateClusterMinimal I0406 18:56:37.718219 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:37.718529 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:37.718577 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:37.718633 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:37.718684 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:37.718727 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:37.718748 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:37.718780 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:37.718804 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:37.718885 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.718923 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.779414 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:37.779590 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:37.779965 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:37.791737 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:37.791919 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:37.791954 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:37.791982 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:37.792010 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:37.792033 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:37.792049 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:37.792070 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:37.792086 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:37.792140 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.792163 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.849702 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:37.849871 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:37.850140 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:37.858205 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:37.858385 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:37.858426 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:37.858456 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:37.858489 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:37.858516 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:37.858532 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:37.858557 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:37.858574 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:37.858640 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.858674 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.956656 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:37.956835 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:37.957144 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:37.962880 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:37.963023 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:37.963058 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:37.963092 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:37.963124 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:37.963153 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:37.963173 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:37.963198 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:37.963216 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:37.963277 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:37.963304 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.023658 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.023830 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.024130 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.031852 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.032117 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.032170 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.032223 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.032273 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.032316 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.032355 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.032392 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.032423 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.032518 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.032566 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.077536 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.077812 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.078172 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.084070 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.084245 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.084277 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.084320 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.084358 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.084383 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.084400 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.084424 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.084440 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.084496 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.084522 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.118031 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.118290 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.118716 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.127957 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.128184 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.128228 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.128271 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.128319 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.128360 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.128391 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.128434 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.128462 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.128595 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.128644 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.153532 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.153694 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.153966 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.160502 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.160736 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.160783 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.160827 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.160877 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.160924 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.160960 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.161004 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.161036 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.161121 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.161169 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.270407 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.270505 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.270815 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.56s) === RUN TestCreateClusterHetzner I0406 18:56:38.276542 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.276735 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.276770 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.276801 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.276836 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.276866 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.276882 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.276908 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.276926 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.276998 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.277027 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 18:56:38.300018 37873 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0406 18:56:38.300055 37873 new_cluster.go:1279] Cloud Provider ID = hetzner I0406 18:56:38.306355 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.306480 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.306507 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.306530 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.306554 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.306577 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.306595 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.306614 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.306626 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.306673 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.306699 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.353000 37873 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.08s) === RUN TestCreateClusterOverride I0406 18:56:38.359860 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.360019 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.360053 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.360081 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.360113 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.360137 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.360153 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.360175 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.360191 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.360257 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.360282 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.428939 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.429440 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.429733 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.08s) === RUN TestCreateClusterComplex I0406 18:56:38.435220 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.435381 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.435415 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.435444 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.435476 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.435502 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.435520 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.435544 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.435562 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.435628 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.435656 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.480068 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.480347 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.480775 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.05s) === RUN TestCreateClusterHA I0406 18:56:38.489748 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.489973 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.490023 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.490069 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.490118 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.490164 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.490197 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.490239 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.490274 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.490382 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.490427 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.530792 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.531400 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.531427 37873 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 18:56:38.531444 37873 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0406 18:56:38.531893 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.544838 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.545013 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.545049 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.545079 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.545112 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.545136 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.545152 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.545178 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.545195 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.545251 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.545275 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.687996 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.688306 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.688321 37873 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 18:56:38.688331 37873 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0406 18:56:38.688635 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.21s) === RUN TestCreateClusterHAGCE I0406 18:56:38.698282 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.698467 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.698505 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.698533 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.698566 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.698591 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.698607 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.698628 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.698643 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.698707 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.698733 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.768819 37873 new_cluster.go:1279] Cloud Provider ID = gce I0406 18:56:38.768950 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.08s) === RUN TestCreateClusterGCE I0406 18:56:38.777195 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.777392 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.777429 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.777457 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.777488 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.777515 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.777532 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.777554 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.777570 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.777635 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.777661 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.823317 37873 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0406 18:56:38.823382 37873 new_cluster.go:1279] Cloud Provider ID = gce I0406 18:56:38.823444 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.05s) === RUN TestCreateClusterHASharedZone I0406 18:56:38.829086 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.829253 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.829287 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.829314 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.829345 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.829372 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.829393 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.829416 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.829433 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.829499 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.829525 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 18:56:38.864994 37873 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0406 18:56:38.865049 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.865279 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.865587 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.04s) === RUN TestCreateClusterHASharedZones I0406 18:56:38.872464 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.872625 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.872661 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.872691 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.872723 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.872751 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.872770 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.872799 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.872816 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.872882 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.872907 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0406 18:56:38.907381 37873 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0406 18:56:38.907435 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.907811 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.907827 37873 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 18:56:38.908144 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.04s) === RUN TestCreateClusterPrivate I0406 18:56:38.917296 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.917447 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.917477 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.917505 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.917536 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.917564 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.917583 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.917605 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.917620 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.917687 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.917711 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.954904 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:38.955092 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:38.955108 37873 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 18:56:38.955354 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:38.955381 37873 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.05s) === RUN TestCreateClusterPrivateGCE I0406 18:56:38.963505 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:38.963657 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:38.963688 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:38.963716 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:38.963744 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:38.963767 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:38.963783 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:38.963803 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:38.963817 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:38.963876 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:38.963902 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.018935 37873 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0406 18:56:39.019004 37873 new_cluster.go:1279] Cloud Provider ID = gce I0406 18:56:39.019095 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.06s) === RUN TestCreateClusterWithNGWSpecified I0406 18:56:39.025881 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.026057 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.026091 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.026119 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.026152 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.026181 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.026198 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.026220 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.026237 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.026305 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.026330 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.121258 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.121478 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:39.121495 37873 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 18:56:39.121815 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:39.121847 37873 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.10s) === RUN TestCreateClusterWithINGWSpecified I0406 18:56:39.130118 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.130293 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.130329 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.130358 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.130391 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.130417 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.130432 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.130454 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.130470 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.130535 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.130560 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.209837 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.210074 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:39.210091 37873 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 18:56:39.210389 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:39.210418 37873 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.09s) === RUN TestCreateClusterSharedVPC I0406 18:56:39.216707 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.216900 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.216939 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.216967 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.216999 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.217023 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.217040 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.217064 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.217082 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.217146 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.217172 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.271403 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.271555 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.271606 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.271646 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.271662 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.271699 37873 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0406 18:56:39.271979 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.06s) === RUN TestCreateClusterSharedSubnets I0406 18:56:39.277204 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.277386 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.277418 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.277448 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.277479 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.277505 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.277520 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.277541 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.277558 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.277625 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.277650 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.353424 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.353483 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.353564 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.353731 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.353760 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.353788 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.353806 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.354212 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.08s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0406 18:56:39.360430 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.360613 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.360652 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.360683 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.360722 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.360754 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.360774 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.360799 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.360817 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.360894 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.360926 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.389490 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:56:39.389545 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.389565 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.389617 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.389733 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.389758 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.389782 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.389799 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.390008 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.04s) === RUN TestCreateClusterPrivateSharedSubnets I0406 18:56:39.395756 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.395918 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.395954 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.395982 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.396015 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.396039 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.396055 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.396077 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.396094 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.396189 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.396220 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.446767 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.446801 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.446862 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.446883 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.446911 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.447042 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.447064 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.447085 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:56:39.447100 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0406 18:56:39.447386 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.06s) === RUN TestCreateClusterIPv6 I0406 18:56:39.454367 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.454586 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.454634 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.454678 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.454725 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.454765 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.454797 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.454833 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.454861 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.454957 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.455000 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.490804 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.491069 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:39.491792 37873 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.05s) === RUN TestCreateClusterDifferentAMIs I0406 18:56:39.500995 37873 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0406 18:56:39.501010 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.501277 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.501313 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.501366 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.501414 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.501445 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.501484 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.501516 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.501541 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.501641 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.501679 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.531781 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.531837 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:39.531850 37873 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0406 18:56:39.532066 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:39.538738 37873 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.04s) === RUN TestCreateClusterKarpenter I0406 18:56:39.538798 37873 featureflag.go:160] FeatureFlag "Karpenter"=true I0406 18:56:39.538809 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.538938 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.538965 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.538991 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.539021 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.539044 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.539063 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.539084 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.539098 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.539153 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.539178 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0406 18:56:39.640911 37873 new_cluster.go:1279] Cloud Provider ID = aws I0406 18:56:39.641091 37873 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0406 18:56:39.641106 37873 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0406 18:56:39.641116 37873 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0406 18:56:39.641517 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:39.653264 37873 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.11s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0406 18:56:39.653862 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.654136 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.654206 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.654342 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.654437 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.654501 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.654570 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.654641 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.654687 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.663593 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0406 18:56:39.665446 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:39.665728 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:39.665786 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:39.665861 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:39.665943 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:39.666075 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:39.666138 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:39.666204 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:39.666252 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:39.690332 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:39.692868 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:39.692966 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 18:56:42.916463 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:45.516395 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0406 18:56:45.516893 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:45.521603 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:45.521630 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:45.521746 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:56:45.521814 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:45.521855 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:45.522967 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:56:45.526846 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:56:45.527482 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:45.527713 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:45.528335 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:45.528859 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:45.528928 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:56:45.529235 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:56:45.529458 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:56:45.554741 37873 target.go:112] Terraform output is in /tmp/test2287174218/out --- PASS: TestMinimal (5.89s) === RUN TestMinimal_v1_23 I0406 18:56:45.557163 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:45.557315 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:45.557355 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:45.557389 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:45.557422 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:45.557482 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:45.557501 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:45.557525 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:45.557544 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:45.572109 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0406 18:56:45.573490 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:45.573548 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:46.909011 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:47.034590 37873 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0406 18:56:47.035202 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:47.035475 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:47.035670 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:47.035786 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:47.035880 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:47.036010 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:47.040590 37873 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0406 18:56:47.046091 37873 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0406 18:56:47.046555 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:47.046934 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:47.048321 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:47.049905 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:47.050023 37873 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0406 18:56:47.050336 37873 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0406 18:56:47.050552 37873 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0406 18:56:47.109016 37873 target.go:112] Terraform output is in /tmp/test3647043912/out --- PASS: TestMinimal_v1_23 (1.56s) === RUN TestMinimal_v1_24 I0406 18:56:47.113409 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:47.113575 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:47.113615 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:47.113687 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:47.113720 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:47.113749 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:47.113767 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:47.113790 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:47.113806 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:47.129228 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:56:47.131060 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:47.131120 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.339255 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:48.449784 37873 executor.go:111] Tasks: 0 done / 93 total; 46 can run W0406 18:56:48.450474 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:48.450534 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:48.450600 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:48.450563 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:48.450675 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:48.450765 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:48.453198 37873 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0406 18:56:48.458052 37873 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0406 18:56:48.458318 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.458457 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.459002 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.459921 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.459989 37873 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0406 18:56:48.460217 37873 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 18:56:48.460397 37873 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 18:56:48.492210 37873 target.go:112] Terraform output is in /tmp/test4261187395/out --- PASS: TestMinimal_v1_24 (1.38s) === RUN TestMinimal_v1_25 I0406 18:56:48.494980 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:48.495126 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:48.495165 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:48.495196 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:48.495228 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:48.495285 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:48.495303 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:48.495325 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:48.495342 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:48.510531 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:56:48.512433 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:48.512491 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:49.934816 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:50.124431 37873 executor.go:111] Tasks: 0 done / 93 total; 46 can run W0406 18:56:50.127179 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:50.128701 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:50.128738 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:50.128817 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:56:50.130402 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:50.131032 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:50.133179 37873 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0406 18:56:50.142645 37873 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0406 18:56:50.143094 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:50.143355 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:50.144276 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:50.146392 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:50.146536 37873 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0406 18:56:50.146846 37873 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0406 18:56:50.147017 37873 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0406 18:56:50.201341 37873 target.go:112] Terraform output is in /tmp/test1425521163/out --- PASS: TestMinimal_v1_25 (1.71s) === RUN TestMinimal_v1_26 I0406 18:56:50.208741 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.209008 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.209059 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.209109 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.209157 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.209243 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.209274 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.209310 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.209335 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.234661 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:56:50.236864 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:50.236951 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:51.706620 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:51.867043 37873 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0406 18:56:51.867311 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:51.867440 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:51.867475 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:51.867542 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:56:51.867723 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:51.868304 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:51.870231 37873 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0406 18:56:51.875365 37873 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0406 18:56:51.875750 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:51.875931 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:51.877045 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:51.877471 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:51.877550 37873 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0406 18:56:51.877795 37873 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0406 18:56:51.877937 37873 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0406 18:56:51.932115 37873 target.go:112] Terraform output is in /tmp/test826631264/out --- PASS: TestMinimal_v1_26 (1.73s) === RUN TestHetzner I0406 18:56:51.936623 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:56:53.193230 37873 executor.go:111] Tasks: 0 done / 50 total; 39 can run I0406 18:56:53.195788 37873 executor.go:111] Tasks: 39 done / 50 total; 5 can run I0406 18:56:53.196850 37873 executor.go:111] Tasks: 44 done / 50 total; 2 can run I0406 18:56:53.202455 37873 executor.go:111] Tasks: 46 done / 50 total; 4 can run I0406 18:56:53.204161 37873 executor.go:111] Tasks: 50 done / 50 total; 0 can run I0406 18:56:53.214376 37873 target.go:112] Terraform output is in /tmp/test3016457157/out --- PASS: TestHetzner (1.28s) === RUN TestNvidia I0406 18:56:53.218402 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:53.218715 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:53.218797 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:53.218883 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:53.218971 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:53.219086 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:53.219147 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:53.219217 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:53.219272 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:53.246318 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:53.248346 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.248408 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.291563 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:53.367638 37873 executor.go:111] Tasks: 0 done / 88 total; 44 can run I0406 18:56:53.367926 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:53.368013 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:53.367948 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:56:53.368133 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:53.368082 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:53.368175 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:53.369757 37873 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0406 18:56:53.372803 37873 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0406 18:56:53.373018 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.373157 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.374021 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.375546 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.375653 37873 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0406 18:56:53.375922 37873 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0406 18:56:53.376079 37873 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0406 18:56:53.408181 37873 target.go:112] Terraform output is in /tmp/test2242283930/out --- PASS: TestNvidia (0.19s) === RUN TestMinimalGossip I0406 18:56:53.410831 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:53.411013 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:53.411063 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:53.411112 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:53.411166 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:53.411251 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:53.411286 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:53.411326 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:53.411353 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:53.427932 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.428013 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.541214 37873 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0406 18:56:53.541501 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:53.543162 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:53.544026 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:53.546168 37873 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0406 18:56:53.549848 37873 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0406 18:56:53.550222 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.550435 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.550854 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.552015 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.552128 37873 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0406 18:56:53.552537 37873 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 18:56:53.552710 37873 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 18:56:53.592341 37873 target.go:112] Terraform output is in /tmp/test568770882/out --- PASS: TestMinimalGossip (0.18s) === RUN TestMinimalGossipIRSA I0406 18:56:53.594887 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:53.595072 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:53.595119 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:53.595159 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:53.595200 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:53.595266 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:53.595291 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:53.595318 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:53.595336 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:53.617978 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.618042 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.719872 37873 executor.go:111] Tasks: 0 done / 89 total; 44 can run W0406 18:56:53.720399 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:53.720517 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:53.720521 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:53.722118 37873 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0406 18:56:53.725247 37873 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0406 18:56:53.725511 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.725636 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.726128 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.729176 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:53.729318 37873 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0406 18:56:53.730693 37873 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0406 18:56:53.730867 37873 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0406 18:56:53.759797 37873 target.go:112] Terraform output is in /tmp/test1089275314/out --- PASS: TestMinimalGossipIRSA (0.17s) === RUN TestMinimalGCE I0406 18:56:53.763686 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:53.765346 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0406 18:56:54.627638 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:54.627703 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:54.627785 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0406 18:56:54.627808 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0406 18:56:54.627823 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:56:54.627844 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:56:54.627911 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:56:54.628970 37873 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0406 18:56:54.633606 37873 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0406 18:56:54.636607 37873 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0406 18:56:54.638388 37873 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0406 18:56:54.638726 37873 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0406 18:56:54.656559 37873 target.go:112] Terraform output is in /tmp/test764367874/out --- PASS: TestMinimalGCE (0.90s) === RUN TestMinimalGCEPrivate I0406 18:56:54.659051 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:54.660255 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:56:55.493213 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:55.493328 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:55.493502 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0406 18:56:55.493532 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0406 18:56:55.493568 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0406 18:56:55.493605 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0406 18:56:55.493701 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:56:55.495408 37873 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0406 18:56:55.497497 37873 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0406 18:56:55.500619 37873 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0406 18:56:55.502757 37873 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0406 18:56:55.503515 37873 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0406 18:56:55.503614 37873 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0406 18:56:55.521886 37873 target.go:112] Terraform output is in /tmp/test1090140826/out --- PASS: TestMinimalGCEPrivate (0.86s) === RUN TestMinimalGCEInternalLoadBalancer I0406 18:56:55.524086 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:55.525333 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:56:55.683074 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:55.683131 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:55.683210 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0406 18:56:55.683245 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0406 18:56:55.683259 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0406 18:56:55.683273 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0406 18:56:55.683315 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0406 18:56:55.683343 37873 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0406 18:56:55.684446 37873 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0406 18:56:55.689388 37873 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0406 18:56:55.692374 37873 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0406 18:56:55.692397 37873 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0406 18:56:55.693961 37873 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0406 18:56:55.694823 37873 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0406 18:56:55.695076 37873 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0406 18:56:55.695231 37873 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0406 18:56:55.713774 37873 target.go:112] Terraform output is in /tmp/test3042842089/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.19s) === RUN TestMinimalGCELongClusterName I0406 18:56:55.716061 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:55.717203 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0406 18:56:55.911837 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:55.911913 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:55.911998 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0406 18:56:55.912020 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0406 18:56:55.912035 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0406 18:56:55.912056 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0406 18:56:55.912103 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:56:55.913147 37873 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0406 18:56:55.915359 37873 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0406 18:56:55.918501 37873 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0406 18:56:55.924338 37873 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0406 18:56:55.924514 37873 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0406 18:56:55.956161 37873 target.go:112] Terraform output is in /tmp/test23357009/out --- PASS: TestMinimalGCELongClusterName (0.24s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0406 18:56:55.958882 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:55.960211 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0406 18:56:56.133805 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:56.133890 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:56.133987 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0406 18:56:56.134009 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0406 18:56:56.134026 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0406 18:56:56.134048 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0406 18:56:56.134091 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0406 18:56:56.134122 37873 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0406 18:56:56.135232 37873 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0406 18:56:56.140946 37873 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0406 18:56:56.143967 37873 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0406 18:56:56.144004 37873 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0406 18:56:56.145742 37873 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0406 18:56:56.145872 37873 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0406 18:56:56.145988 37873 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0406 18:56:56.146078 37873 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0406 18:56:56.166249 37873 target.go:112] Terraform output is in /tmp/test1515887814/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.21s) === RUN TestHA I0406 18:56:56.168809 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:56.168980 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:56.169020 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:56.169052 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:56.169085 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:56.169144 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:56.169162 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:56.169184 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:56.169200 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:56.190360 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:56.192599 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.192661 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.192697 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.192729 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.229987 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:56.308189 37873 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0406 18:56:56.308551 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:56.308753 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:56:56.308819 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:56.308808 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:56.308779 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 18:56:56.308865 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:56.308870 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:56:56.308908 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:56.310462 37873 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0406 18:56:56.316745 37873 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0406 18:56:56.316942 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.317156 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.317344 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.317521 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.318183 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.318442 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.318555 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.318643 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.318720 37873 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0406 18:56:56.318935 37873 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0406 18:56:56.319088 37873 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0406 18:56:56.365424 37873 target.go:112] Terraform output is in /tmp/test1962327587/out --- PASS: TestHA (0.20s) === RUN TestHighAvailabilityGCE I0406 18:56:56.369977 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:56.371395 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:56:56.530085 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:56:56.530160 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:56:56.530242 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0406 18:56:56.530263 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0406 18:56:56.530277 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0406 18:56:56.530306 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0406 18:56:56.530353 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0406 18:56:56.530401 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0406 18:56:56.530438 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:56:56.531934 37873 executor.go:111] Tasks: 0 done / 83 total; 48 can run I0406 18:56:56.534202 37873 executor.go:111] Tasks: 48 done / 83 total; 20 can run I0406 18:56:56.538728 37873 executor.go:111] Tasks: 68 done / 83 total; 7 can run I0406 18:56:56.543998 37873 executor.go:111] Tasks: 75 done / 83 total; 5 can run I0406 18:56:56.545286 37873 executor.go:111] Tasks: 80 done / 83 total; 3 can run I0406 18:56:56.545468 37873 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0406 18:56:56.588463 37873 target.go:112] Terraform output is in /tmp/test3283586113/out --- PASS: TestHighAvailabilityGCE (0.22s) === RUN TestComplex I0406 18:56:56.593543 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:56.593771 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:56.593826 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:56.593937 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:56.593992 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:56.594042 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:56.594078 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:56.594121 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:56.594151 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:56.620573 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:56:56.624496 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:56.624611 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:57.981047 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:58.131528 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0406 18:56:58.133975 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:58.134854 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:58.135413 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:58.135444 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:58.135526 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:58.137214 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:56:58.144799 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:56:58.146741 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.149331 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.149726 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.150796 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.151020 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:56:58.151686 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:56:58.151889 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0406 18:56:58.196691 37873 target.go:112] Terraform output is in /tmp/test2482499340/out I0406 18:56:58.199435 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:58.199573 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:58.199624 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:58.199657 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:58.199690 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:58.199717 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:58.199735 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:58.199758 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:58.199775 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:58.224322 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:56:58.226797 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.226859 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.273813 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:58.439211 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0406 18:56:58.439667 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:58.439760 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:58.439785 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:58.439877 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:56:58.441928 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:58.445138 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:56:58.448207 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:56:58.448509 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.448631 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.449280 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.452250 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.452366 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:56:58.452643 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:56:58.452824 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0406 18:56:58.504680 37873 target.go:112] Terraform output is in /tmp/test2690128423/out --- PASS: TestComplex (1.92s) === RUN TestCompress I0406 18:56:58.509397 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:58.509643 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:58.509692 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:58.509736 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:58.509784 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:58.509865 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:58.509897 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:58.509933 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:58.509958 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:58.529050 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:58.531044 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.531113 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.569142 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:58.659346 37873 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0406 18:56:58.659620 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:58.659810 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:58.660004 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:58.660027 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:58.660119 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:58.661527 37873 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0406 18:56:58.665156 37873 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0406 18:56:58.665411 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.665608 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.670419 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.670620 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.670695 37873 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0406 18:56:58.671034 37873 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0406 18:56:58.671238 37873 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0406 18:56:58.700573 37873 target.go:112] Terraform output is in /tmp/test1650863524/out --- PASS: TestCompress (0.19s) === RUN TestExternalPolicies I0406 18:56:58.703051 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:58.703212 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:58.703253 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:58.703291 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:58.703326 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:58.703387 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:58.703406 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:58.703429 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:58.703447 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:58.720305 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:58.724515 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.724655 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.761623 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:58.877923 37873 executor.go:111] Tasks: 0 done / 100 total; 45 can run I0406 18:56:58.878204 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:58.878234 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:58.878345 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:58.878289 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:58.880653 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:58.880790 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:58.883176 37873 executor.go:111] Tasks: 45 done / 100 total; 20 can run I0406 18:56:58.886201 37873 executor.go:111] Tasks: 65 done / 100 total; 30 can run I0406 18:56:58.886536 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.886713 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.887457 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.888339 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.888417 37873 executor.go:111] Tasks: 95 done / 100 total; 3 can run I0406 18:56:58.888737 37873 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0406 18:56:58.888868 37873 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0406 18:56:58.941239 37873 target.go:112] Terraform output is in /tmp/test1102730002/out --- PASS: TestExternalPolicies (0.24s) === RUN TestMinimalIPv6 I0406 18:56:58.943937 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:58.944102 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:58.944143 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:58.944212 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:58.944246 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:58.944275 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:58.944294 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:58.944319 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:58.944337 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:58.961301 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:58.963445 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:58.963508 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.001055 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:59.120582 37873 executor.go:111] Tasks: 0 done / 106 total; 45 can run W0406 18:56:59.121089 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:56:59.121126 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:59.121315 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:59.121340 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:59.121427 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:59.121433 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:56:59.122978 37873 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0406 18:56:59.126092 37873 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0406 18:56:59.126332 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.126525 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.127208 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.129237 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.129389 37873 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0406 18:56:59.129821 37873 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0406 18:56:59.130063 37873 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 18:56:59.170991 37873 target.go:112] Terraform output is in /tmp/test4021746858/out --- PASS: TestMinimalIPv6 (0.23s) === RUN TestMinimalIPv6Private I0406 18:56:59.173897 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:59.174064 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:59.174108 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:59.174143 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:59.174178 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:59.174249 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:59.174266 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:59.174291 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:59.174309 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:59.199554 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0406 18:56:59.201839 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.201901 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.239696 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:59.387775 37873 executor.go:111] Tasks: 0 done / 124 total; 46 can run W0406 18:56:59.388279 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:59.388264 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:59.388336 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:59.388364 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:56:59.388383 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:59.388462 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:59.390125 37873 executor.go:111] Tasks: 46 done / 124 total; 23 can run I0406 18:56:59.393265 37873 executor.go:111] Tasks: 69 done / 124 total; 36 can run I0406 18:56:59.393411 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.393742 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.394175 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.395200 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.395273 37873 executor.go:111] Tasks: 105 done / 124 total; 11 can run I0406 18:56:59.396309 37873 executor.go:111] Tasks: 116 done / 124 total; 8 can run I0406 18:56:59.396848 37873 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 18:56:59.444866 37873 target.go:112] Terraform output is in /tmp/test1885002433/out --- PASS: TestMinimalIPv6Private (0.27s) === RUN TestMinimalIPv6Calico I0406 18:56:59.447621 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:59.447776 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:59.447817 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:59.447897 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:59.447934 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:59.447961 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:59.447977 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:59.448001 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:59.448019 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:59.464058 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:56:59.467145 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.467239 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.537585 37873 zones.go:93] ListHostedZonesPages { } I0406 18:56:59.894267 37873 executor.go:111] Tasks: 0 done / 109 total; 47 can run I0406 18:56:59.894789 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:56:59.894923 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:59.894955 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:56:59.895771 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:56:59.894981 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:56:59.896747 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:56:59.899563 37873 executor.go:111] Tasks: 47 done / 109 total; 21 can run I0406 18:56:59.902583 37873 executor.go:111] Tasks: 68 done / 109 total; 31 can run I0406 18:56:59.902762 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.903039 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.904104 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.904372 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.904464 37873 executor.go:111] Tasks: 99 done / 109 total; 5 can run I0406 18:56:59.904757 37873 executor.go:111] Tasks: 104 done / 109 total; 5 can run I0406 18:56:59.904944 37873 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0406 18:56:59.942328 37873 target.go:112] Terraform output is in /tmp/test560672386/out --- PASS: TestMinimalIPv6Calico (0.50s) === RUN TestMinimalIPv6Cilium I0406 18:56:59.947264 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:59.947460 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:59.947499 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:59.947531 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:59.947563 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:59.947615 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:59.947633 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:59.947655 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:59.947672 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:59.963424 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:56:59.965532 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:56:59.965588 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.002492 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:00.145087 37873 executor.go:111] Tasks: 0 done / 107 total; 46 can run W0406 18:57:00.145453 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:00.145519 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.145735 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:00.145757 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:00.146209 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:00.147429 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:00.150652 37873 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0406 18:57:00.153888 37873 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0406 18:57:00.154307 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.154499 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.155599 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.155983 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.156061 37873 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0406 18:57:00.156325 37873 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0406 18:57:00.156575 37873 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 18:57:00.193753 37873 target.go:112] Terraform output is in /tmp/test4027793713/out --- PASS: TestMinimalIPv6Cilium (0.25s) === RUN TestMinimalWarmPool I0406 18:57:00.196545 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:00.196708 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:00.196750 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:00.196788 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:00.196822 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:00.196885 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:00.196904 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:00.196930 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:00.196949 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:00.222163 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:00.225597 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.225679 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.262193 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:00.390592 37873 executor.go:111] Tasks: 0 done / 89 total; 45 can run I0406 18:57:00.391148 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:00.391243 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:00.391264 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.391192 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:00.391292 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:00.391109 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:00.392876 37873 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0406 18:57:00.399927 37873 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0406 18:57:00.400309 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.400418 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.401624 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.401814 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.401911 37873 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0406 18:57:00.402224 37873 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0406 18:57:00.402380 37873 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0406 18:57:00.442821 37873 target.go:112] Terraform output is in /tmp/test2239544811/out --- PASS: TestMinimalWarmPool (0.25s) === RUN TestMinimalEtcd I0406 18:57:00.447276 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:00.447468 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:00.447508 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:00.447541 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:00.447575 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:00.447638 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:00.447655 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:00.447677 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:00.447693 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:00.463622 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:00.465464 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.465523 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.507122 37873 zones.go:93] ListHostedZonesPages { } W0406 18:57:00.567603 37873 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0406 18:57:00.567652 37873 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W0406 18:57:00.568810 37873 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0406 18:57:00.579052 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0406 18:57:00.579573 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:00.579646 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.579698 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:00.579684 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:00.579720 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:00.579809 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:00.581506 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:00.584583 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:00.584894 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.585116 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.585805 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.587778 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.587896 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:00.588161 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:00.588304 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:00.622710 37873 target.go:112] Terraform output is in /tmp/test1542788845/out --- PASS: TestMinimalEtcd (0.18s) === RUN TestMinimalGp3 I0406 18:57:00.625183 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:00.625340 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:00.625379 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:00.625410 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:00.625443 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:00.625501 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:00.625518 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:00.625539 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:00.625556 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:00.640280 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:00.642038 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.642094 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.680626 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:00.761070 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0406 18:57:00.761678 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:00.761714 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:00.761767 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.761706 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:00.761802 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:00.761687 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.763092 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:00.768465 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:00.768842 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.769032 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.769469 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.770363 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.770425 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:00.770603 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:00.770707 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:00.799183 37873 target.go:112] Terraform output is in /tmp/test1367262012/out --- PASS: TestMinimalGp3 (0.18s) === RUN TestMinimalLongClusterName I0406 18:57:00.803406 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:00.803594 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:00.803640 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:00.803677 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:00.803712 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:00.803776 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:00.803793 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:00.803817 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:00.803834 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:00.818706 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:00.820471 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.820533 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 18:57:00.859481 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:00.933372 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0406 18:57:00.933572 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:00.933668 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:00.933899 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:00.934076 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:00.934107 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:00.934200 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:00.935497 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:00.938403 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:00.938646 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.938730 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.939398 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.939956 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.940021 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:00.940224 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:00.940345 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:00.973533 37873 target.go:112] Terraform output is in /tmp/test562257582/out --- PASS: TestMinimalLongClusterName (0.17s) === RUN TestExistingSG I0406 18:57:00.976113 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:00.976265 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:00.976304 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:00.976337 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:00.976370 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:00.976426 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:00.976444 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:00.976466 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:00.976483 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:00.995624 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:00.997609 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.997668 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.997702 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:00.997734 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.039730 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:01.115644 37873 executor.go:111] Tasks: 0 done / 146 total; 53 can run I0406 18:57:01.115863 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:01.115879 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:01.115937 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:01.117248 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.118387 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:01.118499 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.118570 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.119087 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.120854 37873 executor.go:111] Tasks: 53 done / 146 total; 26 can run I0406 18:57:01.124090 37873 executor.go:111] Tasks: 79 done / 146 total; 58 can run I0406 18:57:01.124488 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.124647 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.124826 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.125035 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.126154 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.128868 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.129035 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.129292 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.129427 37873 executor.go:111] Tasks: 137 done / 146 total; 5 can run I0406 18:57:01.129737 37873 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0406 18:57:01.129936 37873 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0406 18:57:01.189185 37873 target.go:112] Terraform output is in /tmp/test3096281673/out --- PASS: TestExistingSG (0.22s) === RUN TestBastionAdditionalUserData I0406 18:57:01.192331 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:01.192486 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:01.192526 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:01.192560 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:01.192594 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:01.192653 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:01.192674 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:01.192699 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:01.192717 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:01.210174 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:01.212130 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.212190 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.212227 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.252362 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:01.333593 37873 executor.go:111] Tasks: 0 done / 122 total; 48 can run W0406 18:57:01.333852 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.333967 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.334276 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.334374 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:01.334400 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:01.334353 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:01.334468 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:01.335884 37873 executor.go:111] Tasks: 48 done / 122 total; 29 can run I0406 18:57:01.341667 37873 executor.go:111] Tasks: 77 done / 122 total; 36 can run I0406 18:57:01.341826 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.342052 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.342234 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.342536 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.343145 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.343543 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.343614 37873 executor.go:111] Tasks: 113 done / 122 total; 5 can run I0406 18:57:01.343875 37873 executor.go:111] Tasks: 118 done / 122 total; 4 can run I0406 18:57:01.344042 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0406 18:57:01.387567 37873 target.go:112] Terraform output is in /tmp/test2076155226/out --- PASS: TestBastionAdditionalUserData (0.20s) === RUN TestPrivateWeave I0406 18:57:01.390382 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:01.390542 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:01.390583 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:01.390614 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:01.390649 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:01.390711 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:01.390728 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:01.390751 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:01.390769 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:01.409644 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:01.411517 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.411580 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.411615 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.448664 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:01.532677 37873 executor.go:111] Tasks: 0 done / 120 total; 47 can run I0406 18:57:01.532904 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:01.532975 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:01.532997 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 18:57:01.533095 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.533101 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:01.533123 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.533218 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.534989 37873 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0406 18:57:01.538000 37873 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0406 18:57:01.538272 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.538465 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.539578 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.539793 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.540564 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.542787 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.542869 37873 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0406 18:57:01.543176 37873 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0406 18:57:01.543367 37873 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0406 18:57:01.587739 37873 target.go:112] Terraform output is in /tmp/test4198957618/out --- PASS: TestPrivateWeave (0.20s) === RUN TestPrivateFlannel I0406 18:57:01.591684 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:01.591886 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:01.591934 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:01.592019 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:01.592064 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:01.592101 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:01.592126 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:01.592160 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:01.592184 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:01.618517 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:01.621496 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.621571 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.621626 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.661489 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:01.827946 37873 executor.go:111] Tasks: 0 done / 123 total; 50 can run I0406 18:57:01.828229 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:01.828254 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:01.828297 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:01.828334 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:01.828425 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.828561 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:01.828568 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:01.830218 37873 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0406 18:57:01.833354 37873 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0406 18:57:01.833801 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.833918 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.833960 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.834087 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.835082 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.835434 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.835507 37873 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0406 18:57:01.835801 37873 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0406 18:57:01.835973 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:01.883729 37873 target.go:112] Terraform output is in /tmp/test1530433078/out --- PASS: TestPrivateFlannel (0.30s) === RUN TestPrivateCalico I0406 18:57:01.888177 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:01.888402 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:01.888450 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:01.888540 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:01.888588 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:01.888626 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:01.888652 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:01.888689 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:01.888713 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:01.916338 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:01.919630 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.919707 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.919763 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:01.959817 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:02.301568 37873 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0406 18:57:02.301792 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:02.301809 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:02.301872 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:02.302647 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:02.304408 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:02.304635 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:02.305588 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:02.306942 37873 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0406 18:57:02.310145 37873 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0406 18:57:02.310372 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.310555 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.310708 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.310806 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.311520 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.312029 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.312106 37873 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0406 18:57:02.312378 37873 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0406 18:57:02.312576 37873 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0406 18:57:02.369166 37873 target.go:112] Terraform output is in /tmp/test2966504093/out --- PASS: TestPrivateCalico (0.48s) === RUN TestPrivateCilium I0406 18:57:02.372745 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:02.372892 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:02.372932 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:02.372963 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:02.372996 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:02.373053 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:02.373070 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:02.373091 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:02.373108 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:02.389675 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:02.391699 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.391757 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.391791 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.428716 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:02.534523 37873 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0406 18:57:02.534741 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:02.536598 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:02.537335 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:02.537437 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:02.537633 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:02.538450 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:02.538667 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:02.539890 37873 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0406 18:57:02.542952 37873 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0406 18:57:02.543108 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.543209 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.543357 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.543508 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.543968 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.545602 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.545698 37873 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0406 18:57:02.546080 37873 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0406 18:57:02.546264 37873 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0406 18:57:02.603869 37873 target.go:112] Terraform output is in /tmp/test3134209010/out --- PASS: TestPrivateCilium (0.23s) === RUN TestPrivateCilium2 I0406 18:57:02.606835 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:02.606985 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:02.607024 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:02.607087 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:02.607119 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:02.607145 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:02.607161 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:02.607184 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:02.607201 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:02.623929 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:02.626055 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.626113 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.626147 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:02.663267 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:03.110212 37873 executor.go:111] Tasks: 0 done / 124 total; 51 can run W0406 18:57:03.110714 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.110717 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:03.110756 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.110763 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:03.110851 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:03.110870 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:03.110993 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.113710 37873 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0406 18:57:03.120545 37873 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0406 18:57:03.121023 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.121166 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.121340 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.121407 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.122712 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.122885 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.122997 37873 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0406 18:57:03.123313 37873 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0406 18:57:03.123524 37873 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0406 18:57:03.176507 37873 target.go:112] Terraform output is in /tmp/test1495740871/out --- PASS: TestPrivateCilium2 (0.57s) === RUN TestPrivateCiliumAdvanced I0406 18:57:03.181122 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:03.181296 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:03.181337 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:03.181369 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:03.181402 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:03.181466 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:03.181483 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:03.181508 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:03.181525 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:03.212485 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:03.216468 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.216563 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.216626 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.255804 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:03.357290 37873 executor.go:111] Tasks: 0 done / 126 total; 53 can run W0406 18:57:03.357636 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:03.357764 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.357899 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:03.360077 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:03.360338 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.357926 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:03.361511 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:03.362666 37873 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0406 18:57:03.366214 37873 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0406 18:57:03.366500 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.366630 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.366802 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.366876 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.367326 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.368257 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.368332 37873 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0406 18:57:03.368739 37873 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0406 18:57:03.369062 37873 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0406 18:57:03.416043 37873 target.go:112] Terraform output is in /tmp/test3588904332/out --- PASS: TestPrivateCiliumAdvanced (0.24s) === RUN TestPrivateCanal I0406 18:57:03.420437 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:03.420670 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:03.420722 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:03.420825 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:03.420876 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:03.420920 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:03.420950 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:03.420987 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:03.421015 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:03.448532 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:03.451699 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.451786 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.451845 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.488964 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:03.822245 37873 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0406 18:57:03.822608 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.822612 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:03.822721 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:03.822847 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:03.822972 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:03.822994 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:03.823071 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:03.824451 37873 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0406 18:57:03.827630 37873 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0406 18:57:03.827952 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.828095 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.828156 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.828270 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.830166 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.831994 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:03.832080 37873 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0406 18:57:03.832298 37873 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0406 18:57:03.832472 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:03.878267 37873 target.go:112] Terraform output is in /tmp/test2271512816/out --- PASS: TestPrivateCanal (0.46s) === RUN TestPrivateKopeio I0406 18:57:03.883299 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:03.883529 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:03.883583 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:03.883629 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:03.883675 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:03.883758 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:03.883788 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:03.883826 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:03.883853 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:03.909914 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:03.911835 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:03.911892 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:03.911929 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:03.948995 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:04.027793 37873 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0406 18:57:04.028002 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:04.028019 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:04.028078 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:04.028227 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.028245 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:04.028322 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.028455 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.033504 37873 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0406 18:57:04.036629 37873 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0406 18:57:04.036968 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.037080 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.037121 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.037297 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.037734 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.038664 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.038742 37873 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0406 18:57:04.039071 37873 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0406 18:57:04.039226 37873 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0406 18:57:04.107920 37873 target.go:112] Terraform output is in /tmp/test3984973309/out --- PASS: TestPrivateKopeio (0.23s) === RUN TestUnmanaged I0406 18:57:04.110848 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:04.111045 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:04.111084 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:04.111119 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:04.111158 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:04.111228 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:04.111246 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:04.111270 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:04.111288 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:04.129165 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:04.131108 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.131173 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.131220 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.168550 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:04.280822 37873 executor.go:111] Tasks: 0 done / 108 total; 45 can run W0406 18:57:04.281140 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.281327 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:04.281348 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:04.281404 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:04.281403 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.281496 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.281534 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:04.283194 37873 executor.go:111] Tasks: 45 done / 108 total; 25 can run I0406 18:57:04.286469 37873 executor.go:111] Tasks: 70 done / 108 total; 31 can run I0406 18:57:04.286797 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.286920 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.287073 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.287190 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.287546 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.288101 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.288181 37873 executor.go:111] Tasks: 101 done / 108 total; 4 can run I0406 18:57:04.288496 37873 executor.go:111] Tasks: 105 done / 108 total; 3 can run I0406 18:57:04.288661 37873 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0406 18:57:04.332263 37873 target.go:112] Terraform output is in /tmp/test3621947789/out --- PASS: TestUnmanaged (0.22s) === RUN TestPrivateSharedSubnet I0406 18:57:04.334861 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:04.335021 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:04.335062 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:04.335096 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:04.335130 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:04.335191 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:04.335208 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:04.335230 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:04.335248 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:04.352374 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:04.354266 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.354325 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.354360 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.394665 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:04.474791 37873 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0406 18:57:04.475040 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:04.475093 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.475340 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.475343 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:04.475372 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:04.475515 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:04.475449 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.477209 37873 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0406 18:57:04.477609 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:04.481423 37873 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0406 18:57:04.481868 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.482044 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.482254 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.482344 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.484700 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.485257 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.486033 37873 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0406 18:57:04.486397 37873 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0406 18:57:04.486605 37873 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 18:57:04.532989 37873 target.go:112] Terraform output is in /tmp/test1206223691/out --- PASS: TestPrivateSharedSubnet (0.20s) === RUN TestPrivateSharedIP I0406 18:57:04.535613 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:04.535765 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:04.535803 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:04.535835 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:04.535867 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:04.535922 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:04.535939 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:04.535960 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:04.535977 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:04.552260 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:04.554104 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.554156 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.554193 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.589877 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:04.659568 37873 executor.go:111] Tasks: 0 done / 116 total; 45 can run W0406 18:57:04.659963 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.660077 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.660090 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.660165 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:04.660187 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:04.660139 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:04.660250 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:04.661844 37873 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0406 18:57:04.662180 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:04.664745 37873 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0406 18:57:04.665043 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.665146 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.665297 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.665413 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.666249 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.668677 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:04.668761 37873 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0406 18:57:04.669121 37873 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0406 18:57:04.669348 37873 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 18:57:04.737322 37873 target.go:112] Terraform output is in /tmp/test3039652234/out --- PASS: TestPrivateSharedIP (0.20s) === RUN TestPrivateDns1 I0406 18:57:04.739997 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:04.740154 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:04.740195 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:04.740228 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:04.740263 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:04.740324 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:04.740341 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:04.740364 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:04.740382 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:04.759997 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.760058 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.760094 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.862864 37873 executor.go:111] Tasks: 0 done / 120 total; 46 can run W0406 18:57:04.863431 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.863470 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:04.863535 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:04.863551 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:04.867988 37873 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0406 18:57:04.868223 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0406 18:57:04.868246 37873 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0406 18:57:04.868296 37873 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0406 18:57:04.868306 37873 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0406 18:57:04.868441 37873 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0406 18:57:04.868838 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.868969 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.871736 37873 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0406 18:57:04.871802 37873 api.go:53] PageSize not implemented W0406 18:57:04.871912 37873 api.go:53] PageSize not implemented I0406 18:57:04.874563 37873 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0406 18:57:04.874668 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.874812 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.875449 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.876367 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.876441 37873 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0406 18:57:04.876732 37873 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0406 18:57:04.876867 37873 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0406 18:57:04.935448 37873 target.go:112] Terraform output is in /tmp/test2298386338/out --- PASS: TestPrivateDns1 (0.20s) === RUN TestPrivateDns2 I0406 18:57:04.938210 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:04.938360 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:04.938399 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:04.938431 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:04.938463 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:04.938517 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:04.938534 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:04.938555 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:04.938572 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:04.959131 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.959191 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:04.959225 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.053210 37873 executor.go:111] Tasks: 0 done / 116 total; 44 can run W0406 18:57:05.053444 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:05.053743 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:05.053831 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:05.053922 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:05.058376 37873 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0406 18:57:05.058594 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:05.058679 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0406 18:57:05.058717 37873 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0406 18:57:05.058783 37873 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0406 18:57:05.058803 37873 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0406 18:57:05.058851 37873 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0406 18:57:05.058948 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.059041 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.060725 37873 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0406 18:57:05.060752 37873 api.go:53] PageSize not implemented W0406 18:57:05.060776 37873 api.go:53] PageSize not implemented I0406 18:57:05.063442 37873 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0406 18:57:05.063534 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.063680 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.064674 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.065669 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.065731 37873 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0406 18:57:05.065931 37873 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0406 18:57:05.066031 37873 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 18:57:05.110823 37873 target.go:112] Terraform output is in /tmp/test3529149692/out --- PASS: TestPrivateDns2 (0.18s) === RUN TestDiscoveryFeatureGate I0406 18:57:05.113591 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:05.113755 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:05.113798 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:05.113869 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:05.113902 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:05.113931 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:05.113947 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:05.113972 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:05.113988 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:05.131948 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:05.133903 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.133965 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.174799 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:05.380413 37873 executor.go:111] Tasks: 0 done / 99 total; 51 can run I0406 18:57:05.380692 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:05.380721 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:05.380784 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:05.380813 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:05.380937 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:05.381013 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:05.385474 37873 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0406 18:57:05.391261 37873 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0406 18:57:05.391441 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.391659 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.392513 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.393303 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.393370 37873 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0406 18:57:05.393581 37873 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0406 18:57:05.393696 37873 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0406 18:57:05.434974 37873 target.go:112] Terraform output is in /tmp/test3760883661/out --- PASS: TestDiscoveryFeatureGate (0.32s) === RUN TestVFSServiceAccountIssuerDiscovery I0406 18:57:05.438267 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:05.438416 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:05.438453 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:05.438483 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:05.438516 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:05.438574 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:05.438591 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:05.438613 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:05.438632 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:05.452974 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:05.454682 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.454737 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.491827 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:05.563250 37873 executor.go:111] Tasks: 0 done / 90 total; 45 can run W0406 18:57:05.563460 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:05.563538 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:05.563573 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:05.564013 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:05.564038 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:05.564129 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:05.566505 37873 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0406 18:57:05.571000 37873 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0406 18:57:05.571358 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.571570 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.572049 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.572891 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.572959 37873 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0406 18:57:05.573147 37873 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0406 18:57:05.573270 37873 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0406 18:57:05.606121 37873 target.go:112] Terraform output is in /tmp/test1732873035/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.17s) === RUN TestAWSLBController I0406 18:57:05.608666 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:05.608822 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:05.608862 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:05.608895 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:05.608932 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:05.608993 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:05.609011 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:05.609035 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:05.609053 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:05.624174 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:05.626083 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.626139 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:05.662524 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:06.139407 37873 executor.go:111] Tasks: 0 done / 96 total; 49 can run W0406 18:57:06.140065 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:06.140299 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:06.140316 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:06.140332 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:06.140427 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:06.140528 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:06.148372 37873 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0406 18:57:06.154299 37873 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0406 18:57:06.154791 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.155053 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.156588 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.157666 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.157785 37873 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0406 18:57:06.158125 37873 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0406 18:57:06.158418 37873 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0406 18:57:06.221872 37873 target.go:112] Terraform output is in /tmp/test3870615338/out --- PASS: TestAWSLBController (0.62s) === RUN TestManyAddons I0406 18:57:06.230084 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:06.230308 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:06.230364 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:06.230459 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:06.230506 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:06.230545 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:06.230571 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:06.230606 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:06.230629 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:06.256299 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:06.259668 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.259756 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.298543 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:06.948067 37873 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0406 18:57:06.948614 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:06.948833 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:06.948874 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:06.948942 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:06.949000 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:06.949130 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:06.955844 37873 executor.go:111] Tasks: 50 done / 95 total; 19 can run I0406 18:57:06.959597 37873 executor.go:111] Tasks: 69 done / 95 total; 22 can run I0406 18:57:06.959971 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.960165 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.961445 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.962293 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:06.962507 37873 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0406 18:57:06.962919 37873 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0406 18:57:06.963132 37873 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0406 18:57:07.022194 37873 target.go:112] Terraform output is in /tmp/test854461893/out --- PASS: TestManyAddons (0.80s) === RUN TestManyAddonsCCMIRSA I0406 18:57:07.030432 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:07.030675 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:07.030732 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:07.030836 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:07.030893 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:07.030940 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:07.030971 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:07.031013 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:07.031045 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:07.056405 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:07.058595 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.058653 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.095852 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:07.813633 37873 executor.go:111] Tasks: 0 done / 111 total; 59 can run W0406 18:57:07.814590 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:07.814635 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:07.814657 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:07.814764 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:07.814847 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:07.814903 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:07.819966 37873 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0406 18:57:07.822855 37873 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0406 18:57:07.823235 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.823506 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.825226 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.826744 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.826893 37873 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0406 18:57:07.827330 37873 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0406 18:57:07.827554 37873 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0406 18:57:07.884947 37873 target.go:112] Terraform output is in /tmp/test1425173711/out --- PASS: TestManyAddonsCCMIRSA (0.86s) === RUN TestManyAddonsCCMIRSA23 I0406 18:57:07.891358 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:07.891560 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:07.891603 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:07.891670 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:07.891707 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:07.891734 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:07.891752 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:07.891776 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:07.891793 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:07.910068 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0406 18:57:07.912206 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.912263 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:07.948885 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:08.637154 37873 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0406 18:57:08.637695 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:08.637997 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:08.639185 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:08.639517 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:08.638025 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:08.639627 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:08.645695 37873 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0406 18:57:08.649646 37873 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0406 18:57:08.650034 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.650205 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.653980 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.655354 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.655445 37873 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0406 18:57:08.655820 37873 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0406 18:57:08.656001 37873 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0406 18:57:08.710868 37873 target.go:112] Terraform output is in /tmp/test686739911/out --- PASS: TestManyAddonsCCMIRSA23 (0.83s) === RUN TestManyAddonsCCMIRSA24 I0406 18:57:08.721350 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:08.721597 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:08.721659 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:08.721776 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:08.721831 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:08.721874 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:08.721903 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:08.721942 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:08.721971 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:08.746757 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:08.749420 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.749489 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:08.788065 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:09.466034 37873 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0406 18:57:09.466695 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:09.466759 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:09.466785 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:09.466878 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:09.466953 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:09.467338 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:09.471765 37873 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0406 18:57:09.477551 37873 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0406 18:57:09.478182 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.478521 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.479699 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.481547 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.481761 37873 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0406 18:57:09.482108 37873 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0406 18:57:09.482315 37873 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0406 18:57:09.537595 37873 target.go:112] Terraform output is in /tmp/test2825755170/out --- PASS: TestManyAddonsCCMIRSA24 (0.83s) === RUN TestManyAddonsCCMIRSA25 I0406 18:57:09.549336 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:09.549603 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:09.549658 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:09.549761 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:09.549816 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:09.549857 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:09.549887 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:09.549927 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:09.549954 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:09.578331 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:09.581787 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.581866 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:09.625508 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:10.398940 37873 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0406 18:57:10.399375 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:10.399549 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:10.399619 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:10.399639 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:10.399579 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:10.399681 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:10.405146 37873 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0406 18:57:10.410161 37873 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0406 18:57:10.410561 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.410726 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.411883 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.412167 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.412248 37873 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0406 18:57:10.412500 37873 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0406 18:57:10.412638 37873 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0406 18:57:10.465065 37873 target.go:112] Terraform output is in /tmp/test4137467516/out --- PASS: TestManyAddonsCCMIRSA25 (0.92s) === RUN TestManyAddonsCCMIRSA26 I0406 18:57:10.473391 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:10.473713 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:10.473795 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:10.473921 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:10.473995 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:10.474059 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:10.474112 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:10.474179 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:10.474229 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:10.491752 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:10.493899 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.493963 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:10.540089 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:11.192704 37873 executor.go:111] Tasks: 0 done / 111 total; 59 can run I0406 18:57:11.193061 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:11.193478 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:11.193513 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 18:57:11.193600 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:11.193605 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:11.193683 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:11.199164 37873 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0406 18:57:11.202825 37873 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0406 18:57:11.203030 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.203160 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.203816 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.205733 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.205851 37873 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0406 18:57:11.206090 37873 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0406 18:57:11.206240 37873 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0406 18:57:11.259688 37873 target.go:112] Terraform output is in /tmp/test2550555241/out --- PASS: TestManyAddonsCCMIRSA26 (0.79s) === RUN TestCCM I0406 18:57:11.266693 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:11.266890 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:11.266930 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:11.267006 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:11.267041 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:11.267068 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:11.267086 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:11.267109 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:11.267126 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:11.285420 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:11.287759 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.287858 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.327014 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:11.856044 37873 executor.go:111] Tasks: 0 done / 96 total; 51 can run W0406 18:57:11.856509 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:11.856613 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:11.856634 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:11.856714 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:11.858636 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:11.858957 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:11.861694 37873 executor.go:111] Tasks: 51 done / 96 total; 19 can run I0406 18:57:11.864609 37873 executor.go:111] Tasks: 70 done / 96 total; 22 can run I0406 18:57:11.864872 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.865071 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.866237 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.867016 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.867095 37873 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0406 18:57:11.867341 37873 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0406 18:57:11.867464 37873 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0406 18:57:11.902484 37873 target.go:112] Terraform output is in /tmp/test2763071967/out --- PASS: TestCCM (0.64s) === RUN TestExternalDNS I0406 18:57:11.908303 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:11.908557 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:11.908609 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:11.908658 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:11.908706 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:11.908793 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:11.908822 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:11.908858 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:11.908886 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:11.923773 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:11.925517 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.925576 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:11.965640 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:12.046068 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0406 18:57:12.046697 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:12.046809 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.046900 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:12.046894 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:12.046925 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:12.047013 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:12.049702 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:12.053958 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:12.054276 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.054452 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.055423 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.055660 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.055730 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:12.055939 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:12.056105 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:12.087561 37873 target.go:112] Terraform output is in /tmp/test2233601776/out --- PASS: TestExternalDNS (0.18s) === RUN TestExternalDNSIRSA I0406 18:57:12.090301 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:12.090466 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:12.090505 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:12.090541 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:12.090577 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:12.090649 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:12.090667 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:12.090691 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:12.090710 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:12.109548 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:12.111488 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.111551 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.148372 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:12.254437 37873 executor.go:111] Tasks: 0 done / 92 total; 46 can run I0406 18:57:12.254650 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:12.254671 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:12.254756 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:12.255041 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:12.255171 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:12.255231 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.256586 37873 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0406 18:57:12.261937 37873 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0406 18:57:12.262274 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.262439 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.263005 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.263694 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.263764 37873 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0406 18:57:12.263941 37873 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0406 18:57:12.264035 37873 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0406 18:57:12.296605 37873 target.go:112] Terraform output is in /tmp/test2296731694/out --- PASS: TestExternalDNSIRSA (0.21s) === RUN TestKarpenter I0406 18:57:12.301404 37873 featureflag.go:160] FeatureFlag "Karpenter"=true I0406 18:57:12.301433 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:12.301606 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:12.301647 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:12.301717 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:12.301749 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:12.301775 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:12.301791 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:12.301814 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:12.301831 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:12.320272 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:12.322321 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.322375 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.322409 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.361919 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:12.505688 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.505787 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.505854 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.578822 37873 executor.go:111] Tasks: 0 done / 108 total; 53 can run I0406 18:57:12.579079 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:12.579105 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.579105 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:12.579201 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:12.579495 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.579504 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:12.581176 37873 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0406 18:57:12.585896 37873 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0406 18:57:12.586185 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.586312 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.586461 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.586608 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.587839 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.588455 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.588606 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.589533 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.590248 37873 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0406 18:57:12.590425 37873 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0406 18:57:12.590576 37873 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0406 18:57:12.632801 37873 target.go:112] Terraform output is in /tmp/test864648938/out I0406 18:57:12.636608 37873 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.34s) === RUN TestSharedSubnet I0406 18:57:12.636712 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:12.636867 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:12.636907 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:12.636940 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:12.636975 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:12.637035 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:12.637052 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:12.637076 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:12.637093 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:12.651745 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:12.653500 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.653559 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.689394 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:12.759036 37873 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0406 18:57:12.759373 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:12.759405 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:12.759472 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:12.759467 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:12.759500 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:12.759707 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.761208 37873 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0406 18:57:12.761538 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:12.764610 37873 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0406 18:57:12.764810 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.764951 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.765604 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.766116 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.766184 37873 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0406 18:57:12.766372 37873 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0406 18:57:12.766496 37873 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0406 18:57:12.805725 37873 target.go:112] Terraform output is in /tmp/test3644354391/out --- PASS: TestSharedSubnet (0.17s) === RUN TestSharedVPC I0406 18:57:12.808114 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:12.808272 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:12.808312 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:12.808346 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:12.808380 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:12.808442 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:12.808459 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:12.808483 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:12.808501 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:12.823257 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:12.825093 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.825156 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.861986 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:12.932794 37873 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0406 18:57:12.933194 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:12.933376 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.933465 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:12.933484 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:12.933546 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:12.933592 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:12.934554 37873 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0406 18:57:12.934683 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:12.937577 37873 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0406 18:57:12.937892 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.938047 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.938641 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.940514 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:12.940649 37873 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0406 18:57:12.941725 37873 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0406 18:57:12.941926 37873 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0406 18:57:12.982544 37873 target.go:112] Terraform output is in /tmp/test1748522606/out --- PASS: TestSharedVPC (0.18s) === RUN TestExistingIAM I0406 18:57:12.986723 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:12.986986 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:12.987048 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:12.987106 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:12.987160 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:12.987258 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:12.987289 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:12.987331 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:12.987359 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:13.020182 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:13.023134 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.023216 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.023270 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.023320 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.061749 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:13.159500 37873 executor.go:111] Tasks: 0 done / 101 total; 51 can run W0406 18:57:13.159910 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:13.159949 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.159981 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:13.160004 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:13.160088 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:13.160454 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.160431 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:13.161947 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.167192 37873 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0406 18:57:13.172484 37873 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0406 18:57:13.172881 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.173058 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.173143 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.173364 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.174279 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.174414 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.174682 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.175597 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:13.175715 37873 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0406 18:57:13.176050 37873 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0406 18:57:13.176238 37873 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0406 18:57:13.234886 37873 target.go:112] Terraform output is in /tmp/test684237933/out --- PASS: TestExistingIAM (0.25s) === RUN TestPhaseNetwork I0406 18:57:13.239775 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:13.240049 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:13.240113 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:13.240164 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:13.240219 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:13.240262 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:13.240290 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:13.240329 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:13.240358 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:13.266433 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:13.269065 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.269140 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.269192 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.307696 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:13.407579 37873 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0406 18:57:13.408055 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:13.408181 37873 executor.go:111] Tasks: 46 done / 119 total; 28 can run I0406 18:57:13.408623 37873 executor.go:111] Tasks: 74 done / 119 total; 36 can run I0406 18:57:13.409119 37873 executor.go:111] Tasks: 110 done / 119 total; 5 can run I0406 18:57:13.409375 37873 executor.go:111] Tasks: 115 done / 119 total; 4 can run I0406 18:57:13.409549 37873 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0406 18:57:13.418335 37873 target.go:112] Terraform output is in /tmp/test2399393806/out --- PASS: TestPhaseNetwork (0.18s) === RUN TestExternalLoadBalancer I0406 18:57:13.419163 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:13.419410 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:13.419464 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:13.419512 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:13.419568 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:13.419664 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:13.419694 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:13.419732 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:13.419762 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:13.449169 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:13.452245 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.452355 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.490734 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:13.562589 37873 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0406 18:57:13.563078 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:13.563393 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.564478 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:13.565730 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:13.565762 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:13.565945 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:13.568708 37873 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0406 18:57:13.571796 37873 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0406 18:57:13.572067 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.572218 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.572886 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.573811 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.573916 37873 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0406 18:57:13.574147 37873 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0406 18:57:13.574258 37873 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 18:57:13.610310 37873 target.go:112] Terraform output is in /tmp/test3654378952/out --- PASS: TestExternalLoadBalancer (0.19s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0406 18:57:13.614133 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:13.614343 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:13.614391 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:13.614432 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:13.614490 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:13.614588 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:13.614619 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:13.614662 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:13.614689 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:13.646445 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:13.649573 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649652 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649708 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649762 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649811 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649856 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.649905 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.699569 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:13.813386 37873 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0406 18:57:13.813947 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.814022 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:13.814126 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:13.814151 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:13.814235 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:13.814244 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:13.814245 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:13.815954 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:13.820304 37873 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0406 18:57:13.825744 37873 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0406 18:57:13.825940 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.826229 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.826368 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.826559 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.827020 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.828723 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.830824 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.833202 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.833347 37873 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0406 18:57:13.833802 37873 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0406 18:57:13.834074 37873 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0406 18:57:13.908928 37873 target.go:112] Terraform output is in /tmp/test1241024430/out --- PASS: TestMixedInstancesASG (0.30s) === RUN TestMixedInstancesSpotASG I0406 18:57:13.914370 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:13.914649 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:13.914710 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:13.914767 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:13.914840 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:13.914939 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:13.914970 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:13.915010 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:13.915039 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:13.946967 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:13.949362 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949453 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949506 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949561 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949606 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949649 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.949691 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:13.988035 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:14.097649 37873 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0406 18:57:14.098458 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:14.098735 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:14.098852 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.098832 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:14.098964 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.098965 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:14.099028 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:14.099159 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:14.101715 37873 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0406 18:57:14.106592 37873 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0406 18:57:14.106918 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.107167 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.107335 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.107595 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.111332 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.112292 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.113283 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.113740 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:14.113880 37873 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0406 18:57:14.114205 37873 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0406 18:57:14.114391 37873 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0406 18:57:14.190144 37873 target.go:112] Terraform output is in /tmp/test61089973/out --- PASS: TestMixedInstancesSpotASG (0.28s) === RUN TestAdditionalObjects I0406 18:57:14.195696 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:14.195970 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:14.196031 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:14.196087 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:14.196143 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:14.196238 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:14.196268 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:14.196308 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:14.196336 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:14.224234 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0406 18:57:14.229537 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.229633 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.268133 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:14.396582 37873 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0406 18:57:14.397228 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.397414 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0406 18:57:14.397566 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.397448 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:14.397574 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:14.397616 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:14.403653 37873 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0406 18:57:14.407401 37873 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0406 18:57:14.407787 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.407991 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.409185 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.410601 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0406 18:57:14.410718 37873 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0406 18:57:14.410996 37873 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0406 18:57:14.411258 37873 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0406 18:57:14.465749 37873 target.go:112] Terraform output is in /tmp/test2306088630/out --- PASS: TestAdditionalObjects (0.27s) === RUN TestContainerd I0406 18:57:14.468480 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:14.468642 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:14.468682 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:14.468715 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:14.468749 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:14.468811 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:14.468829 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:14.468852 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:14.468871 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:14.483818 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:14.485558 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.485617 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.525259 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:14.619845 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0406 18:57:14.620153 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.620426 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:14.620520 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.620539 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:14.620588 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:14.620716 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:14.622532 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:14.630275 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:14.630665 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.630874 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.631953 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.633419 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.633531 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:14.633828 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:14.634000 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:14.673940 37873 target.go:112] Terraform output is in /tmp/test1488813168/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0406 18:57:14.676992 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:14.677175 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:14.677218 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:14.677254 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:14.677296 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:14.677373 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:14.677391 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:14.677413 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:14.677430 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:14.695797 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:14.698908 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.699024 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.736023 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:14.807959 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0406 18:57:14.808447 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.808484 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:14.808512 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:14.808508 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:14.808605 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:14.808655 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.810123 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:14.815013 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:14.815194 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.815374 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.816056 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.817311 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.817396 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:14.817630 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:14.817762 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:14.854810 37873 target.go:112] Terraform output is in /tmp/test103789220/out --- PASS: TestContainerdCustom (0.18s) === RUN TestDockerCustom I0406 18:57:14.857333 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:14.857487 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:14.857527 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:14.857557 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:14.857590 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:14.857647 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:14.857665 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:14.857686 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:14.857703 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:14.872756 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:14.874649 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.874704 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.911960 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:14.983530 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0406 18:57:14.983768 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:14.984007 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.984042 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:14.984064 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:14.984149 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:14.984182 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:14.985689 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:14.988533 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:14.988846 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.989067 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.989682 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.990512 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:14.990581 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:14.991549 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:14.991744 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:15.022152 37873 target.go:112] Terraform output is in /tmp/test209137761/out --- PASS: TestDockerCustom (0.17s) === RUN TestAPIServerNodes I0406 18:57:15.024527 37873 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0406 18:57:15.024562 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:15.024707 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:15.024748 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:15.024783 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:15.024817 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:15.024876 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:15.024893 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:15.024916 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:15.024934 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:15.040881 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0406 18:57:15.042819 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:15.042878 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:15.042913 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.298322 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:16.414887 37873 executor.go:111] Tasks: 0 done / 100 total; 47 can run I0406 18:57:16.415436 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:16.415455 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:16.415519 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:16.415674 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:16.415769 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:16.415777 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:16.415788 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:16.417677 37873 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0406 18:57:16.423563 37873 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0406 18:57:16.423881 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.424034 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.424174 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.424707 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.425413 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.425523 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.425592 37873 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0406 18:57:16.425866 37873 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0406 18:57:16.426038 37873 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0406 18:57:16.467171 37873 target.go:112] Terraform output is in /tmp/test2500872165/out I0406 18:57:16.472231 37873 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.45s) === RUN TestNTHQueueProcessor I0406 18:57:16.472372 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:16.472628 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:16.472685 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:16.472733 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:16.472789 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:16.472877 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:16.472909 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:16.472950 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:16.472979 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:16.498784 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0406 18:57:16.501460 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:16.501533 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:17.853462 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:17.967779 37873 executor.go:111] Tasks: 0 done / 101 total; 45 can run W0406 18:57:17.968059 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target W0406 18:57:17.968320 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:17.968678 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:17.968703 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:17.968795 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:17.968778 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:17.970078 37873 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0406 18:57:17.977145 37873 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0406 18:57:17.977368 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:17.977564 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:17.978356 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:17.978885 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:17.978954 37873 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0406 18:57:17.979175 37873 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0406 18:57:17.979376 37873 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0406 18:57:18.026531 37873 target.go:112] Terraform output is in /tmp/test4184785136/out --- PASS: TestNTHQueueProcessor (1.56s) === RUN TestCustomIRSA I0406 18:57:18.031114 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.031341 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:18.031383 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:18.031416 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:18.031449 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:18.031513 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:18.031546 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:18.031588 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:18.031615 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:18.050105 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0406 18:57:18.053293 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.053361 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.095585 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:18.558775 37873 executor.go:111] Tasks: 0 done / 100 total; 50 can run W0406 18:57:18.559164 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:18.559457 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:18.559505 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0406 18:57:18.559549 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:18.559487 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:18.559674 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0406 18:57:18.564777 37873 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0406 18:57:18.570452 37873 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0406 18:57:18.570804 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.571004 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.571642 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.573141 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.573244 37873 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0406 18:57:18.573474 37873 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0406 18:57:18.573622 37873 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0406 18:57:18.618940 37873 target.go:112] Terraform output is in /tmp/test2710958570/out --- PASS: TestCustomIRSA (0.59s) === RUN TestClusterNameDigit I0406 18:57:18.623382 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.623570 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:18.623612 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:18.623645 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:18.623679 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:18.623736 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:18.623753 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:18.623777 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:18.623795 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:18.640222 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0406 18:57:18.642122 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.642183 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.680041 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:18.758701 37873 executor.go:111] Tasks: 0 done / 95 total; 47 can run W0406 18:57:18.759071 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:18.759149 37873 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0406 18:57:18.759171 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:18.759255 37873 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0406 18:57:18.759314 37873 warmpool.go:124] ASG warm pool is not supported by the terraform target I0406 18:57:18.759366 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:18.761115 37873 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0406 18:57:18.765873 37873 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0406 18:57:18.766202 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.766545 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.767297 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.767851 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.767937 37873 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0406 18:57:18.768192 37873 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0406 18:57:18.768344 37873 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0406 18:57:18.815795 37873 target.go:112] Terraform output is in /tmp/test2982408269/out --- PASS: TestClusterNameDigit (0.20s) === RUN TestLifecycleMinimalAWS I0406 18:57:18.818757 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.818932 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:18.818974 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:18.819006 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:18.819036 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:18.819063 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:18.819079 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:18.819100 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:18.819117 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:18.819155 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0406 18:57:18.833442 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:18.835179 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.835237 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 18:57:18.873641 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:18.952710 37873 iamrole.go:89] ListRoles: { } I0406 18:57:18.953849 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0406 18:57:18.954188 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0406 18:57:18.954289 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0406 18:57:18.954343 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.954454 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0406 18:57:18.954492 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:18.954562 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] } I0406 18:57:18.954615 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:18.954656 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.955949 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.956311 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.956624 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.956900 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:18.957179 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:18.957277 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.958173 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:18.958276 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:18.958314 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:18.958339 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:18.958368 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:18.958391 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.958452 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:18.958476 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:18.958976 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:18.959545 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:18.959678 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0406 18:57:18.959706 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.959800 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 18:57:18.959821 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0406 18:57:18.959903 37873 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0406 18:57:18.959976 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 18:57:18.959990 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:18.960075 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:18.960108 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:18.960135 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:18.960143 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 18:57:18.960235 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:18.960249 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:18.960379 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:18.960416 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:18.960539 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:18.960568 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.960678 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.960730 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:18.960760 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0406 18:57:18.960876 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:18.960935 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:18.960995 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:18.961021 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" } ] } I0406 18:57:18.961073 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:18.961105 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:18.961118 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:18.961243 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:18.961276 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:18.962172 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 18:57:18.964202 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:18.964393 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.964547 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.964655 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.964736 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.964809 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.964894 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.965023 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.965199 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.965276 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.965343 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.965576 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.965727 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:18.965830 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:18.965916 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:18.966021 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" }] }] } I0406 18:57:18.966246 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.966482 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.966721 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.966918 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:18.966979 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0406 18:57:18.967118 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:18.967167 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:18.967219 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:18.967258 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.967357 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.968196 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:18.968437 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:18.968531 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:18.968564 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:18.968586 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:18.968626 37873 dns.go:238] Pre-creating DNS records I0406 18:57:18.968643 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:18.972047 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:18.974079 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:18.974139 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0406 18:57:19.014892 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:19.107388 37873 iamrole.go:89] ListRoles: { } I0406 18:57:19.108450 37873 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0406 18:57:19.108680 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0406 18:57:19.108998 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.109030 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:19.109148 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] }] } I0406 18:57:19.109306 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:19.109369 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:19.109477 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.109572 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:19.109606 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:19.109653 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:19.110893 37873 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0406 18:57:19.111033 37873 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0406 18:57:19.111001 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0406 18:57:19.111130 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0406 18:57:19.111168 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.111346 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.111484 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:19.111579 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:19.111671 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.111817 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:19.111862 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:19.111927 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0406 18:57:19.112090 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:19.115160 37873 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0406 18:57:19.115305 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:19.115439 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:19.115585 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:19.115717 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:19.115841 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:19.115929 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:19.116184 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:19.117510 37873 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0406 18:57:19.117674 37873 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0406 18:57:19.117767 37873 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0406 18:57:19.120395 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:19.120458 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:19.120491 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:19.120526 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:19.120560 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:19.120598 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:19.120629 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:19.120661 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:19.120695 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:19.120731 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:19.120765 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:19.120806 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:19.120844 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:19.120876 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:19.120910 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:19.120951 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:19.120984 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:19.121019 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:19.121055 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:19.121093 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:19.121125 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:19.121157 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:19.121189 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:19.121242 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:19.121278 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:19.121313 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:19.122465 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:19.122493 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:19.122534 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122583 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0406 18:57:19.122637 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122701 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122748 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122804 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122849 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122890 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0406 18:57:19.122941 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.122993 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0406 18:57:19.123035 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0406 18:57:19.123079 37873 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0406 18:57:19.123108 37873 api.go:53] PageSize not implemented I0406 18:57:19.123129 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:19.123158 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:19.123177 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:19.123203 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:19.123220 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:19.123271 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:19.123298 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:19.123342 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:19.123383 37873 iamrole.go:89] ListRoles: { } I0406 18:57:19.123437 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:19.123479 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:19.123512 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:19.124072 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0406 18:57:19.124095 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } autoscaling-group:nodes.minimal.example.com ok I0406 18:57:19.124175 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } iam-instance-profile:masters.minimal.example.com ok I0406 18:57:19.124125 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } I0406 18:57:19.124107 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } W0406 18:57:19.124283 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions route53-record:Z1AFAKE1ZON3YO/A/api.minimal.example.com. ok I0406 18:57:19.124239 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 18:57:19.124325 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } iam-instance-profile:nodes.minimal.example.com ok I0406 18:57:19.124296 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:19.124363 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:19.124426 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0406 18:57:19.124490 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:19.124557 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:19.124649 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:19.124665 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:19.124699 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:19.124749 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0406 18:57:19.124823 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:19.124903 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0406 18:57:19.124939 37873 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0406 18:57:19.124948 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0406 18:57:19.124943 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:19.124976 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0406 18:57:19.124998 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0406 18:57:19.125025 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0406 18:57:19.125032 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0406 18:57:19.125049 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0406 18:57:19.125087 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:19.125135 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:19.125179 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:19.125358 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.31s) === RUN TestLifecycleMinimalOpenstack I0406 18:57:19.125598 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0406 18:57:19.131586 37873 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:57:19.337932 37873 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0406 18:57:19.337983 37873 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0406 18:57:19.353467 37873 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0406 18:57:19.361345 37873 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0406 18:57:19.428395 37873 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0406 18:57:19.430978 37873 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0406 18:57:19.433281 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:19.433341 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:19.433512 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0406 18:57:19.437369 37873 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:57:19.574447 37873 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0406 18:57:19.574495 37873 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0406 18:57:19.586061 37873 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0406 18:57:19.588587 37873 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0406 18:57:19.597259 37873 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0406 18:57:19.598887 37873 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0406 18:57:19.600640 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:19.601015 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0406 18:57:19.601910 37873 executor.go:111] Tasks: 96 done / 96 total; 0 can run Subnet:34a69d6c-069f-4c09-9092-19f65636f8c5 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Network:5e2b3de5-14c7-4771-b998-4fdaae7c5f7b ok Port:72f3839a-9469-4d04-837f-72a5d5b05d18 ok SecurityGroup:c9cce864-a92c-4100-bc2d-2b4e946ab2fe ok Port:14ebba90-fba1-480b-887a-0404bc6ccce4 ok SecurityGroup:f3b7db37-b1f1-4062-8904-c89843bd2800 ok Instance:21dac680-3c57-4d6d-8185-075ab2a8db7e ok SecurityGroup:84f35f2d-ab7b-4cc9-bdb6-16914be1b98b ok ServerGroup:803210ed-0a00-41ca-b194-4ef432f8e6fd ok Instance:a1bc3693-071a-4411-a538-d2fe1082c846 ok Port:3bc0e83a-8516-4147-b329-0b018ddcd6c4 ok Instance:b8db5ec0-6a23-498b-8841-52f498f2de00 ok ServerGroup:88e00266-0001-44a8-ad44-ed29d7eab559 ok --- PASS: TestLifecycleMinimalOpenstack (1.58s) === RUN TestLifecycleMinimalGCE I0406 18:57:20.709875 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0406 18:57:20.711104 37873 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0406 18:57:20.910152 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:57:20.910216 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:57:20.910290 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0406 18:57:20.910310 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0406 18:57:20.910324 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:57:20.910344 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:57:20.910387 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:57:20.911469 37873 executor.go:111] Tasks: 0 done / 64 total; 41 can run W0406 18:57:20.916954 37873 executor.go:139] error running task "ProjectIAMBinding/serviceaccount-control-plane" (9s remaining to succeed): error updating IAM for project testproject: googleapi: Error 404: etag I0406 18:57:20.917019 37873 executor.go:111] Tasks: 40 done / 64 total; 18 can run I0406 18:57:20.920292 37873 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0406 18:57:20.921836 37873 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0406 18:57:20.921973 37873 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0406 18:57:20.922043 37873 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0406 18:57:21.090262 37873 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0406 18:57:21.090353 37873 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0406 18:57:21.090460 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0406 18:57:21.090492 37873 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0406 18:57:21.090518 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:57:21.090558 37873 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0406 18:57:21.090635 37873 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0406 18:57:21.092492 37873 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0406 18:57:21.094616 37873 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0406 18:57:21.097544 37873 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0406 18:57:21.100388 37873 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0406 18:57:21.100564 37873 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0406 18:57:21.101662 37873 gce.go:98] Scanning zones: [us-test1-a] I0406 18:57:21.101897 37873 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0406 18:57:21.101913 37873 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" Disk:d1-etcd-main-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1680807440 ok InstanceTemplate:nodes-minimal-gce-example-com-1680807440 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.39s) === RUN TestLifecycleFloatingIPOpenstack I0406 18:57:21.102779 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0406 18:57:21.107813 37873 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:57:21.262142 37873 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0406 18:57:21.262186 37873 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0406 18:57:21.271655 37873 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0406 18:57:21.285998 37873 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0406 18:57:21.349874 37873 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0406 18:57:21.352021 37873 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0406 18:57:21.353571 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:21.353616 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:21.353731 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0406 18:57:21.360543 37873 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0406 18:57:21.494468 37873 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0406 18:57:21.494505 37873 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0406 18:57:21.505816 37873 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0406 18:57:21.512973 37873 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0406 18:57:21.516867 37873 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0406 18:57:21.519488 37873 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0406 18:57:21.522915 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:21.523333 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0406 18:57:21.523520 37873 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0406 18:57:21.524645 37873 executor.go:111] Tasks: 98 done / 98 total; 0 can run Instance:56be45ee-2303-455f-b8c4-58fc4900cc0e ok Instance:d7a21ccd-26b0-4905-84ef-edee5604e3d2 ok SecurityGroup:bc48cc6e-ee9a-4f49-a832-d2f6d926b635 ok SecurityGroup:784d57fc-c5cb-4207-ae20-ffd00dc49d0f ok Port:72c55fca-d32e-4f25-93fb-63b42d339697 ok Port:f1ee149b-568d-4628-916d-dab723ffe262 ok ServerGroup:a81e7c0e-f4f6-4bd2-8fad-e236c2e88c95 ok Port:3fdb720b-0762-4a04-9ccf-40e0b29a1be7 ok Port:b5342301-cbf4-461e-a9ce-e25a97bafc4c ok Router:de5811aa-1708-4d26-b78b-64171e21f2cb ok ServerGroup:33b678f6-846c-4775-8ba9-9206c5d6b962 ok Instance:85500342-6ae3-4540-b611-72c1246e4636 ok Network:10f32132-7150-4fc2-bd11-3378c180b2cf ok Subnet:6cf6a0a9-e33a-4a61-9821-a11ce7c66bd7 ok Router-IF:6cf6a0a9-e33a-4a61-9821-a11ce7c66bd7 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok SecurityGroup:9c46e7d4-eca0-493d-820c-8b90d23a51b9 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.53s) === RUN TestLifecyclePrivateCalico I0406 18:57:22.630720 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:22.630928 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:22.630978 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:22.631029 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:22.631080 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:22.631119 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:22.631148 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:22.631189 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:22.631260 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:22.631328 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0406 18:57:22.651244 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:22.654337 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:22.654406 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:22.654445 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:22.691470 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:22.991179 37873 iamrole.go:89] ListRoles: { } I0406 18:57:22.992581 37873 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0406 18:57:22.992874 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.992992 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:22.993038 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0406 18:57:22.993075 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:22.993134 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 18:57:22.993193 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:22.993249 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0406 18:57:22.993371 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0406 18:57:22.993415 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:22.993439 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 18:57:22.993476 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:22.993495 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0406 18:57:22.993555 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "Name", Value: "bastions.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0406 18:57:22.993624 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:22.993770 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:22.993913 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:22.993972 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.994138 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:22.994219 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:22.994247 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:22.994272 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:22.994305 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:22.994336 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.994495 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:22.994627 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:22.996368 37873 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0406 18:57:22.996621 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.996795 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:22.996763 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 18:57:22.996980 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0406 18:57:22.997068 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 18:57:22.997212 37873 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 18:57:22.997245 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:22.997467 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0406 18:57:22.997650 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 18:57:22.998025 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0406 18:57:22.997714 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.998198 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 18:57:22.999034 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:22.999085 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:22.998285 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 18:57:22.999674 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 18:57:22.999717 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 18:57:22.999771 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:22.999744 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0406 18:57:22.999780 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 18:57:22.999944 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:23.000001 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:22.999943 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.000029 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0406 18:57:23.000182 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.000261 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0406 18:57:23.000342 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.000417 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.000500 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0406 18:57:23.000380 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 18:57:23.000608 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.000700 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:23.000743 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }], VpcId: "vpc-1" } I0406 18:57:23.000814 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:23.000854 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0406 18:57:23.000913 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:23.000944 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:23.000984 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.001018 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:23.001044 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:23.001072 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.001100 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.001163 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:23.001194 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:23.001244 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.001301 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:23.001334 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:23.001358 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:23.001398 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatecalico.example.com" } ] } I0406 18:57:23.001444 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.001504 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:23.001538 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.001566 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:23.001599 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.001649 37873 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0406 18:57:23.001745 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.001849 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 18:57:23.001878 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } W0406 18:57:23.001897 37873 api.go:53] PageSize not implemented I0406 18:57:23.001905 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0406 18:57:23.001939 37873 api.go:53] PageSize not implemented I0406 18:57:23.001977 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" }] }] } I0406 18:57:23.002055 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 18:57:23.002142 37873 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatecalico.example.com" }] } I0406 18:57:23.002187 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 18:57:23.002206 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 18:57:23.002187 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.002238 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 18:57:23.002276 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:23.002286 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.002324 37873 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatecalico.example.com" }] } I0406 18:57:23.002359 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.002303 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 18:57:23.002375 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 18:57:23.002399 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 18:57:23.002399 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0406 18:57:23.002531 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.002621 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:23.002733 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.002861 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.002943 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.003014 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.003133 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0406 18:57:23.003205 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.003252 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.003296 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.003383 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0406 18:57:23.003391 37873 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.003443 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.003499 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.003544 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.003590 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.003671 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.003711 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 18:57:23.003776 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.003852 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0406 18:57:23.003907 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.003980 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:23.004040 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.004110 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.004190 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:23.004222 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:23.004292 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.004367 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:23.004441 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.004523 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.004602 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.004696 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.004827 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.004940 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.005022 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0406 18:57:23.005106 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.005187 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:23.006256 37873 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0406 18:57:23.006350 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.006358 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.006416 37873 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.006462 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.006513 37873 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0406 18:57:23.006570 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:23.006596 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.006621 37873 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 18:57:23.006670 37873 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0406 18:57:23.006725 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.006824 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 18:57:23.006867 37873 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0406 18:57:23.006892 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.006913 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.006945 37873 dns.go:238] Pre-creating DNS records I0406 18:57:23.006960 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.011083 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0406 18:57:23.013122 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.013173 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.013208 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.052497 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.470532 37873 iamrole.go:89] ListRoles: { } I0406 18:57:23.471983 37873 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0406 18:57:23.474108 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.474252 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:23.474286 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:23.474351 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.474498 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.474760 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.475103 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:23.475173 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.475437 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.475525 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.477448 37873 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0406 18:57:23.477565 37873 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 18:57:23.477594 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 18:57:23.477645 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.477772 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0406 18:57:23.477707 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0406 18:57:23.477849 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:23.477909 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0406 18:57:23.478033 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.478071 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0406 18:57:23.478173 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:23.478200 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] }] } I0406 18:57:23.478283 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0406 18:57:23.478376 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.478410 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] }] } I0406 18:57:23.478490 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:23.478514 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.478586 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0406 18:57:23.478668 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:23.478702 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0406 18:57:23.478779 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.478819 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.478869 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.480683 37873 executor.go:111] Tasks: 78 done / 127 total; 40 can run W0406 18:57:23.480842 37873 api.go:53] PageSize not implemented I0406 18:57:23.480859 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.480943 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0406 18:57:23.480972 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0406 18:57:23.480941 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0406 18:57:23.481017 37873 api.go:53] PageSize not implemented I0406 18:57:23.481038 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.481101 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.481122 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 18:57:23.481090 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.481173 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.481261 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481360 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481421 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.481580 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481660 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481782 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481889 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.481998 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.482066 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:23.482094 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0406 18:57:23.482118 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 18:57:23.483449 37873 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0406 18:57:23.483506 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.483559 37873 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" W0406 18:57:23.483567 37873 api.go:53] PageSize not implemented I0406 18:57:23.483587 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.483607 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.483661 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:23.483659 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } W0406 18:57:23.483689 37873 api.go:53] PageSize not implemented I0406 18:57:23.483702 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.483806 37873 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0406 18:57:23.483845 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.483937 37873 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0406 18:57:23.486157 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:23.486204 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:23.486224 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:23.486243 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:23.486263 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:23.486301 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 18:57:23.486330 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:23.486351 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:23.486370 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:23.486389 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:23.486409 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:23.486429 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:23.486450 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:23.486484 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 18:57:23.486507 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:23.486527 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:23.486545 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:23.486566 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:23.486586 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:23.486604 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:23.486624 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:23.486644 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:23.486666 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:23.486686 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:23.486706 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:23.486726 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:23.486747 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:23.486765 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.486786 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:23.486804 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:23.486822 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:23.486846 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:23.486867 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:23.486887 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:23.487812 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:23.487829 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:23.487854 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.487906 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0406 18:57:23.487953 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.487988 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488024 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488055 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488090 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488133 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0406 18:57:23.488191 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488241 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:23.488267 37873 natgateway.go:117] DescribeNatGateways: { } I0406 18:57:23.488307 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0406 18:57:23.488349 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0406 18:57:23.488401 37873 api.go:53] PageSize not implemented I0406 18:57:23.488417 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0406 18:57:23.488470 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:23.488495 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:23.488514 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:23.488538 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:23.488556 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:23.488609 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.488637 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.488681 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:23.488726 37873 iamrole.go:89] ListRoles: { } I0406 18:57:23.488775 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:23.488823 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:23.488865 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1","rtb-2"] } I0406 18:57:23.488896 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:23.488917 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:bastion.privatecalico.example.com ok autoscaling-config:lt-1 ok autoscaling-group:nodes.privatecalico.example.com ok I0406 18:57:23.489334 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok keypair:key-1 ok I0406 18:57:23.489382 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:23.489380 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0406 18:57:23.489309 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:bastion-privatecalico-exa-hocohm ok route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatecalico.example.com. ok I0406 18:57:23.489425 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.489401 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 18:57:23.489459 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:23.489495 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 18:57:23.489467 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0406 18:57:23.489461 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0406 18:57:23.489524 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 18:57:23.489552 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0406 18:57:23.489574 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 18:57:23.489599 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0406 18:57:23.489523 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0406 18:57:23.489708 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.489727 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-3 ok I0406 18:57:23.489769 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0406 18:57:23.489841 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:23.489890 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0406 18:57:23.489947 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.489955 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:23.489974 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.490002 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0406 18:57:23.490065 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.490081 37873 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0406 18:57:23.490155 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:23.490238 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0406 18:57:23.490236 37873 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I0406 18:57:23.490290 37873 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com elastic-ip:eipalloc-1 ok I0406 18:57:23.490304 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0406 18:57:23.490335 37873 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0406 18:57:23.490304 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.490362 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0406 18:57:23.490410 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0406 18:57:23.490421 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0406 18:57:23.490440 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0406 18:57:23.490378 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } W0406 18:57:23.490483 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.490501 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:23.490460 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0406 18:57:23.490570 37873 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0406 18:57:23.490582 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0406 18:57:23.490530 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:23.490605 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } iam-role:masters.privatecalico.example.com ok I0406 18:57:23.490624 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0406 18:57:23.490735 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.490749 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 18:57:23.490776 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 18:57:23.490809 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:23.490867 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:23.490921 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 18:57:23.490960 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:23.491000 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:23.491177 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.86s) === RUN TestLifecyclePrivateKopeio I0406 18:57:23.491455 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:23.491634 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:23.491663 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:23.491687 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:23.491717 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:23.491743 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:23.491759 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:23.491781 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:23.491796 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:23.491831 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0406 18:57:23.515097 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:23.518149 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.518231 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.518290 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.558252 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.648897 37873 iamrole.go:89] ListRoles: { } I0406 18:57:23.650443 37873 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0406 18:57:23.650606 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.650711 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0406 18:57:23.650825 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] } I0406 18:57:23.652405 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0406 18:57:23.650843 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:23.652499 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "bastions.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 18:57:23.652571 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:23.652698 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0406 18:57:23.652732 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.652800 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 18:57:23.652968 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.653110 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:23.653173 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.653273 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.653322 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:23.653391 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.653570 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.653622 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.654046 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0406 18:57:23.654204 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:23.654427 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:23.654504 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:23.654557 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:23.654860 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:23.654895 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.656113 37873 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0406 18:57:23.656225 37873 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 18:57:23.656191 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.656268 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0406 18:57:23.656260 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.656379 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 18:57:23.656346 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.656408 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 18:57:23.656440 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.656489 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 18:57:23.656544 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:23.656631 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.656671 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 18:57:23.656702 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.656847 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 18:57:23.656922 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 18:57:23.656932 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:23.656993 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:23.656878 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.657032 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0406 18:57:23.657214 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.657346 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:23.657364 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0406 18:57:23.657539 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:23.657567 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.657625 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 18:57:23.657699 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.657703 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0406 18:57:23.657595 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 18:57:23.657857 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.657938 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.658004 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:23.658032 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.658092 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0406 18:57:23.658153 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.658176 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.658210 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0406 18:57:23.658267 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:23.658301 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.658357 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 18:57:23.658389 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:23.658412 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:23.658443 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.658468 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:23.658493 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0406 18:57:23.658539 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:23.658564 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.658585 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0406 18:57:23.658660 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.658696 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:23.658835 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:23.658882 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 18:57:23.658929 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0406 18:57:23.658957 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0406 18:57:23.658988 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.659079 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:23.659132 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:23.659173 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:23.659259 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:23.659300 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:23.659334 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0406 18:57:23.659418 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:23.659463 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.659512 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:23.659555 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" }, { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] } I0406 18:57:23.659644 37873 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] } I0406 18:57:23.659725 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.659773 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:23.660403 37873 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0406 18:57:23.660486 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0406 18:57:23.660549 37873 api.go:53] PageSize not implemented W0406 18:57:23.660572 37873 api.go:53] PageSize not implemented I0406 18:57:23.660560 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.660726 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 18:57:23.660641 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.660818 37873 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "Name", Value: "api.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 18:57:23.660885 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 18:57:23.660924 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 18:57:23.660850 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.660979 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 18:57:23.661082 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.661143 37873 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0406 18:57:23.661210 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.661260 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 18:57:23.661315 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 18:57:23.661214 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" }] }] } I0406 18:57:23.661546 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.661734 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.661829 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:23.661857 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0406 18:57:23.661902 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 18:57:23.661953 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 18:57:23.661991 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.662027 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0406 18:57:23.662051 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.662099 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.662139 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.662228 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.662291 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.662354 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-1" } I0406 18:57:23.662402 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.662517 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.662623 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.662732 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.662812 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.662870 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.663046 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:23.663170 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.663288 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.663383 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.663478 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.663598 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0406 18:57:23.663662 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0406 18:57:23.663687 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0406 18:57:23.663807 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.663944 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-3" } I0406 18:57:23.663996 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:23.664044 37873 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0406 18:57:23.664116 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 18:57:23.664146 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:23.664167 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:23.664231 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 18:57:23.664283 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.664366 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:23.664444 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0406 18:57:23.664523 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0406 18:57:23.664566 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:23.664627 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0406 18:57:23.664697 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 18:57:23.664721 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.664796 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0406 18:57:23.664878 37873 tags.go:42] CreateTags { Resources: ["subnet-4"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0406 18:57:23.664907 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:23.664932 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0406 18:57:23.664956 37873 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0406 18:57:23.664983 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:23.665008 37873 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0406 18:57:23.665027 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:23.665054 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0406 18:57:23.665097 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0406 18:57:23.665125 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0406 18:57:23.665168 37873 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0406 18:57:23.665220 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.665236 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.665316 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.665343 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-1" } I0406 18:57:23.665389 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.665467 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I0406 18:57:23.665505 37873 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0406 18:57:23.665607 37873 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0406 18:57:23.665646 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.665683 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.665732 37873 dns.go:238] Pre-creating DNS records I0406 18:57:23.665756 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.671689 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:23.674610 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.674691 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.674749 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.721391 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.809114 37873 iamrole.go:89] ListRoles: { } I0406 18:57:23.811554 37873 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0406 18:57:23.811885 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:23.812162 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.812300 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.812398 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.812466 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:23.812485 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:23.812514 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.812536 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0406 18:57:23.812614 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.812658 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:23.815054 37873 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0406 18:57:23.815242 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.815501 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0406 18:57:23.815684 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0406 18:57:23.815742 37873 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 18:57:23.815789 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 18:57:23.815729 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] }] } I0406 18:57:23.815894 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0406 18:57:23.816613 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.816680 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0406 18:57:23.816827 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.816874 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.816995 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.817085 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.817248 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0406 18:57:23.817860 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:23.817925 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0406 18:57:23.818085 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.818224 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.818373 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:23.818474 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.818565 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.818657 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:23.818690 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:23.818741 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.818797 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:23.818829 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0406 18:57:23.818864 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:23.821902 37873 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0406 18:57:23.822034 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.822124 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } W0406 18:57:23.822202 37873 api.go:53] PageSize not implemented I0406 18:57:23.822221 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.822227 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.822307 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0406 18:57:23.822286 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0406 18:57:23.822344 37873 api.go:53] PageSize not implemented I0406 18:57:23.822371 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.822468 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.822509 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 18:57:23.822524 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.822585 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0406 18:57:23.822628 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.822684 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.822838 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 18:57:23.822931 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0406 18:57:23.823039 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 18:57:23.823151 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.823247 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.823346 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.823458 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:23.824806 37873 executor.go:111] Tasks: 117 done / 126 total; 6 can run W0406 18:57:23.824875 37873 api.go:53] PageSize not implemented I0406 18:57:23.824889 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.824901 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:23.824976 37873 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0406 18:57:23.824948 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:23.824996 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } W0406 18:57:23.825047 37873 api.go:53] PageSize not implemented I0406 18:57:23.825037 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:23.825064 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8","bastion-privatekopeio-exa-d8ef8e"] } I0406 18:57:23.825211 37873 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0406 18:57:23.825315 37873 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0406 18:57:23.828838 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:23.828907 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:23.828948 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:23.828987 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:23.829023 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:23.829067 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:23.829108 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:23.829143 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:23.829182 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:23.829223 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:23.829274 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:23.829319 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:23.829360 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 18:57:23.829405 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:23.829447 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:23.829487 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:23.829536 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:23.829575 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:23.829619 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0406 18:57:23.829663 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:23.829770 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:23.829850 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:23.829927 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:23.830001 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:23.830076 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:23.830153 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:23.830223 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:23.830305 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 18:57:23.830382 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0406 18:57:23.830460 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:23.830532 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:23.830600 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:23.830675 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:23.830740 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:23.830809 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:23.832749 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:23.832778 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:23.832817 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.832874 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0406 18:57:23.832952 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833019 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833069 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833137 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833183 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833239 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0406 18:57:23.833314 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833388 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:23.833427 37873 natgateway.go:117] DescribeNatGateways: { } I0406 18:57:23.833472 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0406 18:57:23.833514 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0406 18:57:23.833563 37873 api.go:53] PageSize not implemented I0406 18:57:23.833582 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0406 18:57:23.833634 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:23.833661 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:23.833677 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:23.833698 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:23.833714 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:23.833758 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:23.833785 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:23.833829 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:23.833872 37873 iamrole.go:89] ListRoles: { } I0406 18:57:23.833916 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:23.833958 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:23.834003 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-3","rtb-1"] } I0406 18:57:23.834041 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0406 18:57:23.834070 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0406 18:57:23.834648 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0406 18:57:23.834649 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-group:nodes.privatekopeio.example.com ok load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0406 18:57:23.834716 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 18:57:23.834780 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } iam-instance-profile:bastions.privatekopeio.example.com ok I0406 18:57:23.834719 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0406 18:57:23.834834 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.834814 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 18:57:23.834849 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } I0406 18:57:23.834856 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } security-group:sg-4 ok iam-instance-profile:masters.privatekopeio.example.com ok I0406 18:57:23.834879 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok I0406 18:57:23.834898 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok autoscaling-group:bastion.privatekopeio.example.com ok I0406 18:57:23.834953 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } autoscaling-config:lt-2 ok I0406 18:57:23.834819 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.privatekopeio.example.com. ok I0406 18:57:23.835012 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 18:57:23.835078 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0406 18:57:23.835002 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] } ] } W0406 18:57:23.835207 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.835227 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:23.835256 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:23.835313 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 18:57:23.835350 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:23.835399 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:23.835446 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0406 18:57:23.835522 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.835545 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-1 ok I0406 18:57:23.835597 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:23.835711 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } I0406 18:57:23.835714 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } subnet:subnet-3 ok I0406 18:57:23.835765 37873 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0406 18:57:23.835778 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0406 18:57:23.835769 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:23.835799 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0406 18:57:23.835854 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0406 18:57:23.835853 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } I0406 18:57:23.835893 37873 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com subnet:subnet-2 ok I0406 18:57:23.835902 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0406 18:57:23.835928 37873 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0406 18:57:23.835918 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0406 18:57:23.835955 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0406 18:57:23.836007 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0406 18:57:23.835982 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:23.836022 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0406 18:57:23.836057 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0406 18:57:23.836050 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:23.836141 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.836158 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:23.836187 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:23.836219 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0406 18:57:23.836276 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:23.836282 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:23.836332 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:23.836371 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0406 18:57:23.836403 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 18:57:23.836450 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:23.836496 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:23.836680 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.35s) === RUN TestLifecycleIPv6 I0406 18:57:23.836908 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:23.837071 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:23.837102 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:23.837128 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:23.837159 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:23.837185 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:23.837204 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:23.837228 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:23.837245 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:23.837281 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0406 18:57:23.857320 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:23.860464 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.860551 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:23.898172 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.011834 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.013066 37873 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0406 18:57:24.013296 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.013540 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.013552 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.013607 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0406 18:57:24.013520 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0406 18:57:24.013634 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.013702 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.013766 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0406 18:57:24.013776 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.013918 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.014008 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:24.014052 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.014151 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.014231 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:24.014266 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:24.014287 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:24.014303 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:24.014326 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.014349 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.014419 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:24.014448 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.014481 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.014581 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.015350 37873 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0406 18:57:24.015453 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0406 18:57:24.015535 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0406 18:57:24.015565 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.015641 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0406 18:57:24.015591 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 18:57:24.015718 37873 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 18:57:24.015800 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:24.015877 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:24.015915 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.015923 37873 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 18:57:24.015940 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0406 18:57:24.015992 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.016072 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:24.016078 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.016122 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:24.016147 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.016167 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.016228 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.016302 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:24.016335 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.016394 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:24.016466 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.016467 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 18:57:24.016552 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:24.016594 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] } I0406 18:57:24.016640 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:24.016668 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.016731 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:24.016758 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:24.016807 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:24.016832 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:24.016855 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.016907 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0406 18:57:24.016989 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:24.017044 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:24.017067 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] } I0406 18:57:24.016990 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.018402 37873 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0406 18:57:24.018506 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.018620 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.018650 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:24.018744 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.018775 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.018802 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.018903 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.019043 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.019130 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.019211 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:24.019297 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" }] }] } I0406 18:57:24.019441 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.019573 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.019654 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.019735 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 18:57:24.019790 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0406 18:57:24.019897 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" }] }] } I0406 18:57:24.019985 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.020032 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.020083 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.020109 37873 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.020124 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" }] }] } I0406 18:57:24.020207 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0406 18:57:24.020288 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 18:57:24.020313 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.020394 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.020474 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0406 18:57:24.020550 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.020621 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:24.020649 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0406 18:57:24.020703 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:24.020773 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 18:57:24.020801 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.020892 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0406 18:57:24.020972 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.021040 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.021106 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.021212 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.024600 37873 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0406 18:57:24.024710 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.024801 37873 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.024897 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.025000 37873 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0406 18:57:24.025099 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:24.025081 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.025156 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.025167 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0406 18:57:24.025240 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:24.025415 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.025469 37873 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 18:57:24.025517 37873 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "api.minimal-ipv6.example.com" }], Type: "network" } I0406 18:57:24.025883 37873 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0406 18:57:24.025964 37873 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 18:57:24.026022 37873 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0406 18:57:24.026120 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.026139 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.026313 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 18:57:24.026241 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.027029 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.027123 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.027198 37873 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 18:57:24.027234 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.027254 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.027286 37873 dns.go:238] Pre-creating DNS records I0406 18:57:24.027301 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.030846 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:24.032873 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.032937 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.072114 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.206488 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.208492 37873 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0406 18:57:24.208911 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.208954 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.209071 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.209140 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:24.209162 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:24.209192 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.209215 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.209299 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.209338 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.209408 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.210947 37873 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0406 18:57:24.211163 37873 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 18:57:24.211090 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.211253 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 18:57:24.211220 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0406 18:57:24.211297 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0406 18:57:24.211261 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0406 18:57:24.211410 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:24.211456 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.211540 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0406 18:57:24.211685 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.211732 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.211850 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.211968 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0406 18:57:24.212102 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.212149 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.216139 37873 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0406 18:57:24.216350 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.216455 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:24.216482 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0406 18:57:24.216508 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 18:57:24.216578 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.216638 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.216748 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.216834 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.216955 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.217047 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.217220 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] }] } I0406 18:57:24.217339 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.217400 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.217459 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.218588 37873 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0406 18:57:24.218665 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.218719 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.218698 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.218732 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 18:57:24.218758 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 18:57:24.218781 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 18:57:24.218808 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0406 18:57:24.218760 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:24.218829 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 18:57:24.218879 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.218986 37873 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0406 18:57:24.219033 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.219047 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.219091 37873 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0406 18:57:24.219107 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:24.219103 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.219143 37873 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0406 18:57:24.219121 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 18:57:24.219179 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:24.219194 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 18:57:24.219238 37873 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0406 18:57:24.221004 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:24.221040 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:24.221061 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:24.221080 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:24.221098 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:24.221119 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 18:57:24.221140 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:24.221158 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:24.221176 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:24.221211 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:24.221234 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:24.221254 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:24.221275 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:24.221298 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:24.221318 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:24.221336 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:24.221355 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:24.221376 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:24.221394 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:24.221413 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:24.221433 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:24.221454 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:24.221473 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.221497 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:24.221516 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:24.221536 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:24.221554 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:24.221575 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:24.221596 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:24.221616 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:24.222474 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:24.222492 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:24.222517 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222548 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0406 18:57:24.222583 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222619 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222649 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222680 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222702 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222729 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0406 18:57:24.222760 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222785 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:24.222801 37873 natgateway.go:117] DescribeNatGateways: { } I0406 18:57:24.222825 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0406 18:57:24.222849 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0406 18:57:24.222878 37873 api.go:53] PageSize not implemented I0406 18:57:24.222889 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.222903 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.222911 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0406 18:57:24.222932 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:24.222945 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:24.222956 37873 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1" ] } I0406 18:57:24.222989 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.223005 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.223030 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:24.223051 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.223077 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:24.223106 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:24.223131 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0406 18:57:24.223152 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:24.223171 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:nodes.minimal-ipv6.example.com ok I0406 18:57:24.223939 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.224004 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } iam-instance-profile:nodes.minimal-ipv6.example.com ok autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok I0406 18:57:24.223985 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:24.224028 37873 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0406 18:57:24.223918 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }, { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok route53-record:Z1AFAKE1ZON3YO/AAAA/api.internal.minimal-ipv6.example.com. ok I0406 18:57:24.224101 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:24.224111 37873 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0406 18:57:24.224089 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.224164 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-2 ok I0406 18:57:24.224181 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } autoscaling-config:lt-1 ok iam-instance-profile:masters.minimal-ipv6.example.com ok I0406 18:57:24.224204 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.224241 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 18:57:24.224348 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:24.224362 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:24.224391 37873 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0406 18:57:24.224427 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 18:57:24.224462 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:24.224492 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:24.224526 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:24.224591 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.224626 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 } ] } W0406 18:57:24.224798 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:24.224809 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:24.224828 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:24.224873 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.224907 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0406 18:57:24.224918 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0406 18:57:24.224896 37873 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0406 18:57:24.224955 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0406 18:57:24.224982 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.225017 37873 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0406 18:57:24.224984 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:24.225031 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0406 18:57:24.225056 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0406 18:57:24.225107 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 18:57:24.225186 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:24.225275 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:24.225589 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.39s) === RUN TestLifecycleSharedVPC I0406 18:57:24.225912 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:24.226091 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:24.226122 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:24.226147 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:24.226178 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:24.226206 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:24.226224 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:24.226249 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:24.226265 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:24.226301 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0406 18:57:24.241649 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:24.243433 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.243498 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.282828 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.382779 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.383655 37873 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0406 18:57:24.383809 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.383851 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.383957 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "sharedvpc.example.com" }] }] } I0406 18:57:24.384047 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.383841 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0406 18:57:24.384163 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] } I0406 18:57:24.384239 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.384201 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.384414 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:24.384453 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0406 18:57:24.384497 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:24.384547 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0406 18:57:24.384600 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:24.384655 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.384810 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:24.384861 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.385011 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:24.385732 37873 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0406 18:57:24.385884 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 18:57:24.385976 37873 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 18:57:24.385955 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0406 18:57:24.386031 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 18:57:24.386088 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:24.386143 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:24.386172 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.386131 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 18:57:24.386197 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.386310 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I0406 18:57:24.386331 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:24.386351 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 18:57:24.386445 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.386538 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:24.386637 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0406 18:57:24.386652 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0406 18:57:24.386732 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:24.386775 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] } I0406 18:57:24.386856 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:24.386956 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:24.387003 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.387057 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.387109 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.387274 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 18:57:24.388772 37873 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0406 18:57:24.388921 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.389085 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.389177 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.389263 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 18:57:24.389376 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 18:57:24.389477 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" }] }] } I0406 18:57:24.389602 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:24.389698 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 18:57:24.389834 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.389983 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.390082 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.390145 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.390217 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 18:57:24.390342 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.390466 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 18:57:24.390502 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.390566 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.390640 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.390829 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0406 18:57:24.390961 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.391094 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.391205 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.391292 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:24.391341 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:24.391370 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0406 18:57:24.393478 37873 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0406 18:57:24.393710 37873 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0406 18:57:24.393807 37873 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0406 18:57:24.393844 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.393874 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.393923 37873 dns.go:238] Pre-creating DNS records I0406 18:57:24.393951 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.399586 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:24.402349 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.402441 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.439739 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.535478 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.536561 37873 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0406 18:57:24.536854 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.536930 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:24.537014 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:24.537063 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:24.537084 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:24.537070 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.537114 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.537204 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.538734 37873 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0406 18:57:24.538831 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 18:57:24.538868 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.539053 37873 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 18:57:24.539010 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0406 18:57:24.539171 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.539225 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.539265 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0406 18:57:24.539409 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.539462 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:24.539562 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.541823 37873 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0406 18:57:24.541943 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.542057 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.542163 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.542225 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.542400 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.542545 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.542635 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.543847 37873 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0406 18:57:24.544017 37873 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0406 18:57:24.544153 37873 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0406 18:57:24.546058 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:24.546102 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:24.546126 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:24.546149 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:24.546169 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:24.546191 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:24.546214 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:24.546236 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:24.546260 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:24.546285 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:24.546313 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:24.546334 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:24.546354 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:24.546376 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:24.546397 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:24.546423 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:24.546447 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.546472 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:24.546493 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:24.546515 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:24.546536 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:24.546561 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:24.546584 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:24.547505 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:24.547524 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:24.547550 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547583 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0406 18:57:24.547621 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547659 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547686 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547712 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547742 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547767 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0406 18:57:24.547797 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547832 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0406 18:57:24.547855 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0406 18:57:24.547926 37873 api.go:53] PageSize not implemented I0406 18:57:24.547939 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.547954 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.547965 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:24.547993 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:24.548006 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:24.548044 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.548064 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.548093 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:24.548118 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.548146 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:24.548171 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:24.548190 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.548453 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-group:nodes.sharedvpc.example.com ok I0406 18:57:24.548477 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 18:57:24.548514 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0406 18:57:24.548562 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } volume:vol-1 ok iam-instance-profile:masters.sharedvpc.example.com ok autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok autoscaling-config:lt-1 ok I0406 18:57:24.548597 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.548553 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.548615 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedvpc.example.com. ok I0406 18:57:24.548667 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } iam-instance-profile:nodes.sharedvpc.example.com ok I0406 18:57:24.548631 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0406 18:57:24.548748 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:24.548762 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0406 18:57:24.548787 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:24.548818 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.548843 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:24.548903 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:24.548911 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:24.548959 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0406 18:57:24.548988 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0406 18:57:24.548997 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0406 18:57:24.548986 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:24.549015 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0406 18:57:24.549031 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0406 18:57:24.549053 37873 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0406 18:57:24.549060 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0406 18:57:24.549075 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0406 18:57:24.549107 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:24.549253 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.32s) === RUN TestLifecycleComplex I0406 18:57:24.549431 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:24.549581 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:24.549608 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:24.549633 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:24.549663 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:24.549686 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:24.549705 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:24.549726 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:24.549741 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:24.549774 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0406 18:57:24.573519 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:24.576736 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.576812 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.614204 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.743169 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.744584 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0406 18:57:24.744917 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.744990 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.746986 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.747015 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:24.747172 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 18:57:24.747401 37873 api.go:53] PageSize not implemented I0406 18:57:24.747509 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:24.747657 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.748064 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:24.748725 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:24.748811 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "k8s.io/etcd/main", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "foo/bar", Value: "fib+baz" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.748874 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0406 18:57:24.748947 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.748973 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] } I0406 18:57:24.749053 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:24.749072 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0406 18:57:24.749116 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:24.749146 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:24.749170 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:24.749138 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0406 18:57:24.749206 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.749298 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:24.749594 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.749771 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:24.749915 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:24.751093 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:57:24.751185 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:24.751214 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:24.751320 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:24.751293 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:24.751347 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 18:57:24.751409 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 18:57:24.751371 37873 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "tls-complex-example-com-5nursn" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 18:57:24.751211 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.751488 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 18:57:24.751476 37873 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0406 18:57:24.751548 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.751623 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0406 18:57:24.751690 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:24.751770 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 18:57:24.751999 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.751971 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:24.752115 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:24.752052 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 18:57:24.752161 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.752327 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.752419 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.752502 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.752587 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:24.752685 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.752703 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Public" } ] }], VpcId: "vpc-1" } I0406 18:57:24.752758 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:24.752842 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:24.752692 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 18:57:24.752926 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.752983 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.753064 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.753104 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:24.753140 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:24.753178 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:24.753139 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:24.753300 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 18:57:24.753333 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:24.753294 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 18:57:24.753364 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0406 18:57:24.753484 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:24.753535 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.753569 37873 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0406 18:57:24.753625 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:24.753672 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:24.753703 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0406 18:57:24.753789 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:24.753824 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.753854 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" } ] }], VpcId: "vpc-1" } I0406 18:57:24.753942 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:24.753977 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:24.754025 37873 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0406 18:57:24.754090 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:24.754138 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] } I0406 18:57:24.754216 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:24.754251 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0406 18:57:24.754387 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:57:24.754498 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.754577 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.754679 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.754729 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.754656 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.754804 37873 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-2" }], Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ], Type: "network" } I0406 18:57:24.754871 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-1", TransitGatewayId: "tgw-123456" } I0406 18:57:24.754883 37873 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0406 18:57:24.754945 37873 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0406 18:57:24.754990 37873 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:24.754969 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 18:57:24.755094 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.755178 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 18:57:24.755260 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0406 18:57:24.755356 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.755448 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:24.755475 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:24.755499 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" } ] }] } I0406 18:57:24.755592 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.755674 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 18:57:24.755699 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0406 18:57:24.755737 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0406 18:57:24.755764 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.755860 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 18:57:24.755951 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.756016 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-1", TransitGatewayId: "tgw-0123456" } I0406 18:57:24.756045 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0406 18:57:24.756103 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 18:57:24.756190 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.756283 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0406 18:57:24.756343 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 18:57:24.756434 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 18:57:24.756521 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0406 18:57:24.756578 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 18:57:24.756672 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" } ] }] } I0406 18:57:24.756762 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0406 18:57:24.756815 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0406 18:57:24.756870 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0406 18:57:24.756950 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.757034 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0406 18:57:24.757064 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-3" } I0406 18:57:24.757085 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0406 18:57:24.757138 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0406 18:57:24.757212 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0406 18:57:24.757292 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0406 18:57:24.757428 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0406 18:57:24.757509 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0406 18:57:24.757656 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.757755 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.757841 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0406 18:57:24.758057 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" } ] }] } I0406 18:57:24.758206 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0406 18:57:24.758344 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.758428 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.759626 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:57:24.759680 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.759787 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.759898 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:57:24.759991 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0406 18:57:24.760022 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.760043 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.760075 37873 dns.go:238] Pre-creating DNS records I0406 18:57:24.760090 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:24.764869 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:24.767370 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.767439 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.804398 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.939537 37873 iamrole.go:89] ListRoles: { } I0406 18:57:24.940967 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0406 18:57:24.941214 37873 api.go:53] PageSize not implemented I0406 18:57:24.941341 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:24.941439 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.941524 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:24.941532 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:24.941648 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.941785 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:24.941856 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:24.941916 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.941982 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:24.942000 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:24.942033 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.942062 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.943593 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:57:24.943795 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:24.943721 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.943893 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:24.943878 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.943962 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:24.944039 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:24.944205 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:24.944300 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:24.944317 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.944401 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:24.944502 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:24.944865 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.945077 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.945255 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:24.945311 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:24.945453 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.945739 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.946348 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.946402 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.946442 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:24.946476 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:24.946550 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:24.946569 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:24.946712 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:24.947346 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:24.950128 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:57:24.950261 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.950375 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:24.950441 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:24.950422 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.950462 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:24.950508 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:24.950544 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:24.950571 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:24.950591 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:24.950563 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.950610 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:24.950728 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.950801 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.950892 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.951082 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.951215 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.951391 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:24.951516 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.951643 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:24.952998 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:57:24.953076 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:24.953156 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:24.953179 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:24.953198 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:24.953268 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:57:24.953354 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0406 18:57:24.959288 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:24.962354 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:24.973907 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:24.977644 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:24.977735 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.014689 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.147204 37873 iamrole.go:89] ListRoles: { } I0406 18:57:25.149538 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0406 18:57:25.149803 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:25.149820 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 18:57:25.150055 37873 api.go:53] PageSize not implemented I0406 18:57:25.149995 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:25.150206 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:25.150333 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.150478 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.150614 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:25.150715 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.150819 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:25.150860 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:25.150891 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:25.151022 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.151656 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:57:25.151746 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.151855 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:25.151836 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:25.151841 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.151918 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:25.151912 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.151943 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.151981 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:25.152045 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.152074 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:25.152104 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.152189 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.152243 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.152314 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.152359 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0406 18:57:25.152445 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:25.152471 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0406 18:57:25.152547 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:25.152569 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:25.152669 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:25.152703 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.152752 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:25.152851 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:25.152884 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:25.152968 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:25.154962 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:57:25.155133 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.155212 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.155329 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0406 18:57:25.155383 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0406 18:57:25.155419 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:25.155437 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.155501 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.155503 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.155536 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.155572 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.155606 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.155636 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.155633 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.155754 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.155813 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0406 18:57:25.155846 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.155873 37873 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0406 18:57:25.155900 37873 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0406 18:57:25.155852 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.155927 37873 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0406 18:57:25.156038 37873 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0406 18:57:25.156116 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.156151 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.156076 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.156254 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.156409 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.156642 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.157381 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:57:25.157453 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:25.157511 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:25.157527 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:25.157542 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.157588 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:57:25.157693 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0406 18:57:25.157731 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.157762 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:25.157810 37873 dns.go:238] Pre-creating DNS records I0406 18:57:25.164944 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:25.170040 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.170480 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.207516 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.330769 37873 iamrole.go:89] ListRoles: { } I0406 18:57:25.332205 37873 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0406 18:57:25.332467 37873 api.go:53] PageSize not implemented I0406 18:57:25.332412 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:25.332577 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.332676 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:25.332709 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:25.332773 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:25.332871 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.332975 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.333052 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:25.333113 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.333169 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:25.333189 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:25.333221 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.336258 37873 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0406 18:57:25.336784 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:25.336734 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.336776 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:25.336852 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.336837 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.336940 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.337026 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:25.337117 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.337191 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:25.337184 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:25.337243 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.337292 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:25.337324 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:25.337331 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:25.337357 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:25.337382 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.337430 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0406 18:57:25.337519 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:25.337544 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.337594 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.337638 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.337715 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:25.337738 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:25.337820 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:25.337876 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.340281 37873 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0406 18:57:25.340464 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:25.340513 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:25.340525 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.340554 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.340580 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.340608 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.340635 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.340498 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.340665 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.340760 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.340953 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.341078 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.341184 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.341297 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.341532 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.341655 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.341723 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.341787 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.341918 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.342842 37873 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0406 18:57:25.342929 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:25.343010 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:25.343037 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:25.343061 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.343134 37873 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0406 18:57:25.343222 37873 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0406 18:57:25.345806 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.347911 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.355308 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:25.357750 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.357827 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.395872 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.524420 37873 iamrole.go:89] ListRoles: { } I0406 18:57:25.526574 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0406 18:57:25.526729 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:25.526771 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0406 18:57:25.526866 37873 api.go:53] PageSize not implemented I0406 18:57:25.526863 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.526970 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.527075 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:25.527182 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:25.527284 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:25.527352 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.527456 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:25.527531 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.527600 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:25.527619 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:25.527654 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.528877 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:25.529016 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:25.528952 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:25.529047 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:25.529032 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:25.529062 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.529094 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:25.529170 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.529196 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:25.529216 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.529227 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:25.529275 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.529365 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:25.529388 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.529463 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.529514 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:25.529608 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:25.529647 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.529739 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.529790 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.529865 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:25.529958 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:25.529988 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.530037 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.530111 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:25.530133 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.532035 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:25.532121 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.532146 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:25.532192 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:25.532206 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.532244 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.532279 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.532239 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.532317 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.532348 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.532334 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.532369 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.532494 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.532745 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.532869 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.532951 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.533049 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.533215 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.533292 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.533493 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.535386 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.535506 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.535726 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:25.535810 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:25.535882 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:25.535917 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:25.535944 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.537303 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:25.537411 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:25.537443 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.537465 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:25.537507 37873 dns.go:238] Pre-creating DNS records I0406 18:57:25.542198 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:25.545112 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.545178 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.582721 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.703383 37873 iamrole.go:89] ListRoles: { } I0406 18:57:25.704791 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0406 18:57:25.704944 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:25.704970 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 18:57:25.705148 37873 api.go:53] PageSize not implemented I0406 18:57:25.705084 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 18:57:25.705225 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:25.705319 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] }] } I0406 18:57:25.705435 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:25.705501 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:25.705555 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.706693 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:25.706839 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.706960 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:25.707001 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:25.708619 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.709867 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:25.709989 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:25.709974 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.710089 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:25.710144 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:25.710126 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.710194 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.710218 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.710285 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:25.710347 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.710391 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:25.710434 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.710521 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.710655 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:25.710691 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.710770 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0406 18:57:25.710904 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0406 18:57:25.711028 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:25.711216 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:25.711256 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.711354 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:25.711475 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.711550 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:25.711707 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:25.711765 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:25.711804 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:25.713090 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:25.713160 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.713222 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.713276 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:25.713334 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:25.713346 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.713374 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.713397 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.713421 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.713442 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.713423 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.713461 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.713529 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.713642 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.713762 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.713876 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.713955 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.714066 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.714221 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.714320 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.715998 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:25.716111 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:25.716196 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:25.716217 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:25.716242 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.716291 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:25.716399 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0406 18:57:25.721802 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.725540 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.725614 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.735925 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:25.739942 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.740023 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.740082 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.777457 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.943326 37873 iamrole.go:89] ListRoles: { } I0406 18:57:25.945604 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0406 18:57:25.945804 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.946038 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:25.946009 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0406 18:57:25.946107 37873 api.go:53] PageSize not implemented I0406 18:57:25.946161 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:25.946294 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:25.946334 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:25.946401 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.946451 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:25.946551 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:25.946649 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:25.946739 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:25.946839 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.946988 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.948035 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:25.948132 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.948229 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:25.948256 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:25.948231 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:25.948340 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.948356 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:25.948418 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:25.948418 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:25.948492 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.948506 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.948673 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.948838 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:25.948890 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:25.949072 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:25.949198 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:25.949365 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:25.949417 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0406 18:57:25.949527 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.949601 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.949676 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.949743 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:25.949835 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.949908 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:25.949974 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:25.950016 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:25.953545 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:25.953646 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.953779 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0406 18:57:25.953806 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0406 18:57:25.953864 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:25.953888 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.953936 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.953974 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.954015 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:25.954053 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:25.954028 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.954086 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:25.954125 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:25.954591 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.954758 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.954883 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.954984 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.955058 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:25.955119 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.955205 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.957743 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:25.957894 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:25.957995 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:25.958026 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:25.958060 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:25.958166 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:25.958295 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:25.958341 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:25.958379 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:25.958440 37873 dns.go:238] Pre-creating DNS records I0406 18:57:25.967321 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:25.972412 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.972640 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:25.972733 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.010626 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.207779 37873 iamrole.go:89] ListRoles: { } I0406 18:57:26.209470 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0406 18:57:26.209761 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.209835 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:26.209935 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:26.209981 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:26.210045 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } W0406 18:57:26.210152 37873 api.go:53] PageSize not implemented I0406 18:57:26.210097 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.210230 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.210402 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0406 18:57:26.210550 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:26.210684 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:26.210796 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:26.210909 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:26.211023 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:26.211922 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:26.212033 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:26.212049 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:26.212160 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:26.212162 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:26.212248 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.212177 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:26.212317 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:26.212370 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:26.212378 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.212440 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.213377 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:26.213439 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.213623 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:26.213665 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.213765 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.213918 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:26.214052 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.214135 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.214232 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 18:57:26.214384 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:26.214432 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:26.214610 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:26.214667 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.214752 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0406 18:57:26.214901 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.218212 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:26.218324 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.218475 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.218532 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:26.218613 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:26.218632 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.218635 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.218684 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.218725 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.218766 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.218755 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.218806 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.218841 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.218988 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.219138 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.219368 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.219541 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.219680 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.219894 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.220019 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.221342 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:26.221639 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:26.221710 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:26.221735 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:26.221752 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.221805 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:26.221899 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0406 18:57:26.224234 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.226752 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.226826 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.226956 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.233948 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:26.236812 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.236892 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.236941 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.236971 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.273789 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.440508 37873 iamrole.go:89] ListRoles: { } I0406 18:57:26.441994 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0406 18:57:26.442250 37873 api.go:53] PageSize not implemented I0406 18:57:26.442315 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:26.445273 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:26.445286 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0406 18:57:26.446711 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.446863 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.446980 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.447095 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:26.447187 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:26.447275 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.447366 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:26.447392 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:26.447435 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:26.447552 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.447621 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:26.447710 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:26.447758 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:26.447721 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.447788 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:26.447802 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.447884 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.447948 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:26.447948 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.448011 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.448069 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.448136 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:26.448237 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:26.448275 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:26.448359 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:26.448389 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 18:57:26.448483 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:26.448512 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.448608 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:26.448637 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.448697 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.448760 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.448843 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:26.448942 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:26.449003 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0406 18:57:26.449171 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:26.451268 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:26.451428 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.451552 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.452059 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.452306 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.452424 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.452439 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:26.452530 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:26.452549 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.452569 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.452588 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.452634 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.452668 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.452692 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.452711 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.452753 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.452898 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.453187 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.453299 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.453392 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.454657 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:26.454768 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:26.454830 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:26.454848 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:26.454864 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.454916 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:26.455017 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:26.455053 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.455077 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:26.455114 37873 dns.go:238] Pre-creating DNS records I0406 18:57:26.460380 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0406 18:57:26.462898 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.462962 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.462999 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.463029 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.503108 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.648151 37873 iamrole.go:89] ListRoles: { } I0406 18:57:26.650290 37873 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0406 18:57:26.650455 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:26.650518 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0406 18:57:26.650634 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0406 18:57:26.650697 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.650852 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0406 18:57:26.650936 37873 api.go:53] PageSize not implemented I0406 18:57:26.651004 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.651127 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.651236 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.651314 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:26.651340 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:26.651390 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0406 18:57:26.651519 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0406 18:57:26.651620 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.652490 37873 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0406 18:57:26.652609 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.652698 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.652760 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:26.652704 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0406 18:57:26.652806 37873 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:26.652818 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.652817 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0406 18:57:26.652888 37873 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0406 18:57:26.652957 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.652995 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:26.653067 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.653217 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:26.653264 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.653349 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0406 18:57:26.653493 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:26.653602 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] }] } I0406 18:57:26.653762 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:26.653803 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0406 18:57:26.653958 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:26.654015 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.654149 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.654235 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.654369 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.654510 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0406 18:57:26.654567 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.657273 37873 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0406 18:57:26.657468 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.657524 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:26.657609 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:26.657628 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.657672 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.657662 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.657707 37873 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.657752 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0406 18:57:26.657736 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.657791 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0406 18:57:26.657821 37873 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0406 18:57:26.657989 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.658068 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.658167 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.658303 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.658421 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.658474 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.658615 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.658810 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:26.660877 37873 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0406 18:57:26.660974 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:26.661040 37873 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0406 18:57:26.661061 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0406 18:57:26.661081 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.661145 37873 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0406 18:57:26.661237 37873 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0406 18:57:26.661481 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:26.661520 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:26.661539 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:26.661557 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:26.661577 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:26.661600 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:26.661618 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:26.661637 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:26.661655 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:26.661677 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:26.661700 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:26.661727 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:26.661747 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:26.661766 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:26.661791 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:26.661810 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:26.661832 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:26.661854 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:26.661877 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:26.661900 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:26.661922 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:26.661942 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:26.661967 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0406 18:57:26.662017 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:26.662038 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:26.662057 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:26.662081 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:26.662103 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:26.662124 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:26.663737 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:26.663762 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:26.663784 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.663823 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0406 18:57:26.663862 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.663902 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.663946 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.664002 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.664043 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.664082 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0406 18:57:26.664118 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.664163 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0406 18:57:26.664190 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0406 18:57:26.664221 37873 api.go:53] PageSize not implemented I0406 18:57:26.664235 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:26.664251 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:26.664259 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0406 18:57:26.664285 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:26.664300 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:26.664311 37873 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" ] } I0406 18:57:26.664354 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.664371 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:26.664397 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:26.664419 37873 iamrole.go:89] ListRoles: { } I0406 18:57:26.664446 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:26.664475 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:26.664504 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0406 18:57:26.664885 37873 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0406 18:57:26.664896 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0406 18:57:26.664915 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-config:lt-2 ok I0406 18:57:26.664964 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0406 18:57:26.664953 37873 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } volume:vol-1 ok I0406 18:57:26.664988 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } autoscaling-group:nodes.complex.example.com ok I0406 18:57:26.665034 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0406 18:57:26.665001 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0406 18:57:26.665074 37873 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0406 18:57:26.665108 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:26.664978 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok route53-record:Z1AFAKE1ZON3YO/A/api.complex.example.com ok I0406 18:57:26.665160 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:26.665206 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:26.665245 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 } ] } W0406 18:57:26.665484 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:26.665530 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:26.665616 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:26.665690 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0406 18:57:26.666016 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:26.666053 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:26.666130 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:26.666209 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 18:57:26.666285 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:26.666360 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0406 18:57:26.666516 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:26.666628 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0406 18:57:26.666703 37873 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0406 18:57:26.666746 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0406 18:57:26.666801 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0406 18:57:26.666847 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0406 18:57:26.666848 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 18:57:26.666892 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0406 18:57:26.666931 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0406 18:57:26.666970 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0406 18:57:26.667044 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 18:57:26.667104 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:26.667153 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:26.667443 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.12s) === RUN TestLifecycleExternalLB I0406 18:57:26.667746 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:26.667985 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:26.668028 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:26.668063 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:26.668106 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:26.668144 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:26.668170 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:26.668200 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:26.668217 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:26.668254 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0406 18:57:26.695534 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:26.698625 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.698737 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.739732 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.866622 37873 iamrole.go:89] ListRoles: { } I0406 18:57:26.868404 37873 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0406 18:57:26.868588 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:26.868712 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:26.868683 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:26.868804 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0406 18:57:26.868863 37873 api.go:53] PageSize not implemented I0406 18:57:26.868866 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:26.868840 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } W0406 18:57:26.868957 37873 api.go:53] PageSize not implemented I0406 18:57:26.869085 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:26.869052 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } I0406 18:57:26.869043 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:26.869197 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:26.869170 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0406 18:57:26.869273 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:26.869352 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:26.869282 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:26.869433 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } W0406 18:57:26.869479 37873 api.go:53] PageSize not implemented I0406 18:57:26.869492 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 18:57:26.869458 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0406 18:57:26.869482 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.869529 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0406 18:57:26.869610 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:26.869665 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:26.869722 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:26.869861 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:26.870927 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:26.871039 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:26.871316 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.871445 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:26.871486 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:26.871521 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:26.871576 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.871616 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0406 18:57:26.873782 37873 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0406 18:57:26.873890 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.874374 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:26.874425 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0406 18:57:26.874464 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:26.874523 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 18:57:26.874525 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0406 18:57:26.874594 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 18:57:26.874680 37873 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0406 18:57:26.874677 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:26.874714 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] } I0406 18:57:26.874768 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:26.874783 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 18:57:26.874793 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:26.874871 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:26.874907 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:26.874860 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 18:57:26.874931 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:26.875057 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:26.875069 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0406 18:57:26.875271 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0406 18:57:26.875385 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" }] }] } I0406 18:57:26.875440 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:26.875472 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:26.875559 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:26.875618 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:26.875679 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:26.875725 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }], VpcId: "vpc-1" } I0406 18:57:26.875822 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:26.875868 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:26.876843 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 18:57:26.879044 37873 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0406 18:57:26.879238 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.879413 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:26.879470 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 18:57:26.879680 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.879770 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" }] }] } I0406 18:57:26.879900 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" }] }] } I0406 18:57:26.880046 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.880127 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.880189 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.880267 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.880421 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.880682 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.880924 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I0406 18:57:26.881077 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.881158 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:26.881256 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.881382 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 18:57:26.881550 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.881677 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" }] }] } I0406 18:57:26.881803 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0406 18:57:26.881927 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:26.881978 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 18:57:26.882098 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:26.882141 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:26.882176 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0406 18:57:26.882358 37873 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0406 18:57:26.882564 37873 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0406 18:57:26.882643 37873 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 18:57:26.882681 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:26.882717 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:26.882765 37873 dns.go:238] Pre-creating DNS records I0406 18:57:26.882790 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:26.889177 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:26.891848 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.891936 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:26.931533 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.007542 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.008730 37873 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0406 18:57:27.009012 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.010051 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0406 18:57:27.010536 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:27.010653 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } W0406 18:57:27.010722 37873 api.go:53] PageSize not implemented I0406 18:57:27.010739 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 18:57:27.010881 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0406 18:57:27.010993 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:27.011073 37873 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:27.011215 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.011162 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:27.011449 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:27.011653 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0406 18:57:27.012518 37873 api.go:53] PageSize not implemented W0406 18:57:27.012630 37873 api.go:53] PageSize not implemented I0406 18:57:27.011966 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.013783 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:27.013820 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:27.013868 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.014033 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.014186 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:27.014378 37873 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0406 18:57:27.014509 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0406 18:57:27.014471 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.014588 37873 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0406 18:57:27.014597 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0406 18:57:27.014718 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.014760 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0406 18:57:27.014911 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.014963 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.015079 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.015214 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:27.015251 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:27.015332 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:27.017239 37873 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0406 18:57:27.017332 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.017442 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.017621 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.017731 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.017783 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.017905 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.017994 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.019447 37873 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0406 18:57:27.019673 37873 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0406 18:57:27.019759 37873 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0406 18:57:27.021921 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:27.021967 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:27.021987 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:27.022006 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:27.022024 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:27.022047 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.022066 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:27.022083 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:27.022104 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:27.022124 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:27.022144 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:27.022168 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:27.022192 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:27.022210 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:27.022228 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:27.022248 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:27.022266 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.022284 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.022305 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:27.022326 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.022347 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:27.022365 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:27.022383 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.022404 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.022426 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:27.022445 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:27.023282 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:27.023299 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:27.023325 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023356 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0406 18:57:27.023390 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023425 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023453 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023486 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023511 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023532 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0406 18:57:27.023559 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023589 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0406 18:57:27.023613 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0406 18:57:27.023646 37873 api.go:53] PageSize not implemented I0406 18:57:27.023662 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:27.023678 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:27.023690 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:27.023703 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:27.023714 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:27.023746 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.023765 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.023796 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:27.023821 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.023849 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:27.023874 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:27.023893 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.024238 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-group:nodes.externallb.example.com ok I0406 18:57:27.024326 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.024291 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok I0406 18:57:27.024437 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0406 18:57:27.024376 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } I0406 18:57:27.024501 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } W0406 18:57:27.024580 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.024369 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok I0406 18:57:27.024599 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0406 18:57:27.024598 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } security-group:sg-1 ok iam-instance-profile:masters.externallb.example.com ok I0406 18:57:27.024631 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:27.024689 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:27.024736 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0406 18:57:27.024766 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-1 ok I0406 18:57:27.024829 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.024872 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0406 18:57:27.024992 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.025010 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0406 18:57:27.025057 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0406 18:57:27.025089 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0406 18:57:27.025098 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0406 18:57:27.025117 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0406 18:57:27.025105 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:27.025134 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0406 18:57:27.025183 37873 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0406 18:57:27.025199 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0406 18:57:27.025233 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0406 18:57:27.025303 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:27.025376 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:27.025435 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:27.025689 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.36s) === RUN TestLifecycleSharedSubnet I0406 18:57:27.025985 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:27.026242 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:27.026285 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:27.026324 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:27.026370 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:27.026409 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:27.026441 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:27.026476 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:27.026501 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:27.026554 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0406 18:57:27.051427 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.054218 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.054300 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.093065 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.179124 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.180031 37873 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0406 18:57:27.180306 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0406 18:57:27.180366 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.180399 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0406 18:57:27.180431 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.180554 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.180660 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.180715 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.180799 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0406 18:57:27.180781 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.180914 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.180949 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.181049 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.181120 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.181148 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 18:57:27.181253 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.181297 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.181311 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.182176 37873 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0406 18:57:27.182306 37873 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 18:57:27.182340 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0406 18:57:27.182475 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.182495 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.182581 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.182624 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.182647 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0406 18:57:27.182625 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 18:57:27.183546 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 18:57:27.183586 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.183681 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.183726 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.183778 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 18:57:27.183861 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.183843 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.184087 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.184170 37873 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] } I0406 18:57:27.186117 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 18:57:27.188104 37873 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0406 18:57:27.188280 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 18:57:27.188615 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.188902 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.189180 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.189409 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 18:57:27.189516 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" }] }] } I0406 18:57:27.189653 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.189751 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.189815 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.189910 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.190007 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:27.190100 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.190196 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:27.190280 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0406 18:57:27.190367 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0406 18:57:27.190450 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.190558 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0406 18:57:27.190706 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.192648 37873 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0406 18:57:27.192904 37873 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0406 18:57:27.192992 37873 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0406 18:57:27.193025 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.193054 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.193095 37873 dns.go:238] Pre-creating DNS records I0406 18:57:27.193114 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.197752 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.199587 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.199648 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.236689 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.316610 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.317632 37873 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0406 18:57:27.317780 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.317944 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.317998 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.318020 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.318743 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.319116 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.319326 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.319410 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.322909 37873 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0406 18:57:27.323082 37873 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 18:57:27.323070 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 18:57:27.323192 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 18:57:27.323143 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.323248 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0406 18:57:27.323390 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.323426 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0406 18:57:27.323505 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.325877 37873 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0406 18:57:27.326026 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.326169 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.326365 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.326456 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.327404 37873 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0406 18:57:27.327541 37873 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0406 18:57:27.327603 37873 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0406 18:57:27.329462 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:27.329530 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:27.329566 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:27.329600 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:27.329629 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.329664 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:27.329697 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:27.329732 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:27.329771 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:27.329797 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:27.329821 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:27.329840 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:27.329858 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:27.329876 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.329895 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.329915 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.329937 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:27.329955 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:27.329980 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.330015 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.330051 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:27.330886 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:27.330905 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:27.330929 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.330961 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0406 18:57:27.330995 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331030 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331054 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331077 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331101 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331121 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0406 18:57:27.331143 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331168 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0406 18:57:27.331188 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0406 18:57:27.331232 37873 api.go:53] PageSize not implemented I0406 18:57:27.331247 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:27.331263 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:27.331274 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:27.331288 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:27.331300 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:27.331329 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.331348 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.331378 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:27.331418 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.331450 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:27.331472 37873 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.sharedsubnet.example.com ok I0406 18:57:27.331721 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok I0406 18:57:27.331705 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.sharedsubnet.example.com. ok I0406 18:57:27.331779 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:27.331865 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.331878 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0406 18:57:27.331822 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.331929 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0406 18:57:27.331919 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:27.331976 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 18:57:27.332022 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:27.332054 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.332134 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 18:57:27.332189 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0406 18:57:27.332093 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 18:57:27.332320 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.332330 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:27.332385 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0406 18:57:27.332438 37873 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0406 18:57:27.332451 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0406 18:57:27.332473 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0406 18:57:27.332493 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.332525 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0406 18:57:27.332532 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0406 18:57:27.332550 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0406 18:57:27.332722 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.31s) === RUN TestLifecyclePrivateSharedSubnet I0406 18:57:27.332965 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:27.333160 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:27.333188 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:27.333213 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:27.333242 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:27.333265 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:27.333282 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:27.333302 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:27.333317 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:27.333350 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0406 18:57:27.350623 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.352518 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.352573 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.352607 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.389493 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.467464 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.469671 37873 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0406 18:57:27.469995 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.470109 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.470198 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.470226 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.470211 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.470254 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.470296 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 18:57:27.470477 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.470471 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.470578 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 18:57:27.470609 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.470698 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.470755 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.470819 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 18:57:27.470863 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.470925 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.471000 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.471163 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.472022 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.473779 37873 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0406 18:57:27.473946 37873 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 18:57:27.473921 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.473999 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 18:57:27.474029 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0406 18:57:27.474106 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.474309 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 18:57:27.474408 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.474308 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0406 18:57:27.474518 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.474638 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.474577 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0406 18:57:27.474854 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 18:57:27.475404 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.475447 37873 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0406 18:57:27.475505 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0406 18:57:27.475624 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.475684 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0406 18:57:27.475649 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.475750 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0406 18:57:27.476229 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:27.476263 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.476315 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.476364 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.476400 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.476425 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.476474 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:27.476499 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.476519 37873 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" } ] } I0406 18:57:27.476571 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.476621 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:27.476649 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.476668 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.476694 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.476716 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:27.476742 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.477560 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.478397 37873 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0406 18:57:27.478556 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" }] }] } W0406 18:57:27.478659 37873 api.go:53] PageSize not implemented I0406 18:57:27.478693 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" }] }] } W0406 18:57:27.478829 37873 api.go:53] PageSize not implemented I0406 18:57:27.478868 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0406 18:57:27.478888 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 18:57:27.478977 37873 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "Name", Value: "api.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0406 18:57:27.479040 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 18:57:27.479018 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.479070 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 18:57:27.479157 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 18:57:27.479222 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 18:57:27.479311 37873 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" }] } I0406 18:57:27.479212 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.479378 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 18:57:27.479412 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0406 18:57:27.479468 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0406 18:57:27.479473 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.479643 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 18:57:27.479744 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 18:57:27.479880 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" }] }] } I0406 18:57:27.480024 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.480139 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.480233 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:27.480330 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.480411 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.480497 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.480575 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:27.480646 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 18:57:27.480724 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.480799 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.480855 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.480928 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.481087 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.481226 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.481351 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.481476 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0406 18:57:27.481607 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0406 18:57:27.481732 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.481842 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.481882 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 18:57:27.481989 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0406 18:57:27.482771 37873 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0406 18:57:27.482876 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.482988 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.483118 37873 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0406 18:57:27.483294 37873 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 18:57:27.483340 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.483378 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.483427 37873 dns.go:238] Pre-creating DNS records I0406 18:57:27.483448 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.490266 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.493278 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.493364 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.493418 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.530817 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.605969 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.607322 37873 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0406 18:57:27.607582 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.607552 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.607738 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.607801 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.607930 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] }] } I0406 18:57:27.608028 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.608073 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.608090 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.609607 37873 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0406 18:57:27.609699 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0406 18:57:27.609829 37873 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 18:57:27.609854 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 18:57:27.609882 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 18:57:27.609858 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.609961 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.610018 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0406 18:57:27.610177 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.610214 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0406 18:57:27.610381 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.610436 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0406 18:57:27.610559 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0406 18:57:27.610660 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0406 18:57:27.610802 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.610855 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0406 18:57:27.610999 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.612522 37873 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0406 18:57:27.612645 37873 api.go:53] PageSize not implemented I0406 18:57:27.612672 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0406 18:57:27.612697 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.612757 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0406 18:57:27.612816 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } W0406 18:57:27.612864 37873 api.go:53] PageSize not implemented I0406 18:57:27.612888 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0406 18:57:27.612966 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 18:57:27.612808 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.613002 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 18:57:27.613092 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.613439 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.613557 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.613794 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.615108 37873 executor.go:111] Tasks: 100 done / 107 total; 4 can run W0406 18:57:27.615181 37873 api.go:53] PageSize not implemented I0406 18:57:27.615191 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.615253 37873 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0406 18:57:27.615195 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } W0406 18:57:27.615326 37873 api.go:53] PageSize not implemented I0406 18:57:27.615340 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0406 18:57:27.615433 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0406 18:57:27.615488 37873 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0406 18:57:27.615602 37873 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0406 18:57:27.618716 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:27.618776 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:27.618812 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:27.618845 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:27.618879 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.618911 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:27.618943 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:27.618974 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:27.619009 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:27.619046 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:27.619084 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 18:57:27.619120 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:27.619151 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:27.619181 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:27.619211 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.619249 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.619281 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:27.619313 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:27.619346 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:27.619378 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.619413 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:27.619446 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:27.619480 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.619516 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.619549 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:27.620989 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:27.621013 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:27.621051 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621111 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0406 18:57:27.621181 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621250 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621290 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621330 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621369 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621402 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0406 18:57:27.621438 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621478 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0406 18:57:27.621511 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0406 18:57:27.621551 37873 api.go:53] PageSize not implemented I0406 18:57:27.621567 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0406 18:57:27.621613 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:27.621640 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:27.621657 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:27.621679 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:27.621695 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0406 18:57:27.621739 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.621765 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.621808 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:27.621846 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.621886 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:27.621924 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:27.622406 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.622465 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0406 18:57:27.622476 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok keypair:key-1 ok I0406 18:57:27.622499 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.622423 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } autoscaling-group:nodes.private-shared-subnet.example.com ok I0406 18:57:27.622542 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.622515 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0406 18:57:27.622530 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.private-shared-subnet.example.com. ok iam-instance-profile:masters.private-shared-subnet.example.com ok I0406 18:57:27.622575 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } load-balancer:bastion-private-shared-su-5ol32q ok load-balancer:api-private-shared-subnet-n2f8ak ok autoscaling-group:bastion.private-shared-subnet.example.com ok I0406 18:57:27.622619 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.622582 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0406 18:57:27.622708 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.622727 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:27.622674 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } autoscaling-config:lt-1 ok iam-instance-profile:nodes.private-shared-subnet.example.com ok I0406 18:57:27.622802 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 18:57:27.622852 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.622888 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0406 18:57:27.623022 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.623039 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok autoscaling-config:lt-2 ok I0406 18:57:27.623087 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-3 ok I0406 18:57:27.623145 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.623174 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0406 18:57:27.623231 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.623239 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:27.623304 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.623326 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.623358 37873 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0406 18:57:27.623373 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0406 18:57:27.623405 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } iam-role:masters.private-shared-subnet.example.com ok I0406 18:57:27.623438 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.623373 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0406 18:57:27.623493 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.623509 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:27.623480 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0406 18:57:27.623538 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.623573 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0406 18:57:27.623571 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } I0406 18:57:27.623608 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } W0406 18:57:27.623623 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-role:nodes.private-shared-subnet.example.com ok I0406 18:57:27.623630 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 18:57:27.623639 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.623677 37873 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0406 18:57:27.623685 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0406 18:57:27.623702 37873 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0406 18:57:27.623919 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.29s) === RUN TestLifecyclePrivateSharedIP I0406 18:57:27.624133 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:27.624297 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:27.624326 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:27.624351 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:27.624380 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:27.624406 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:27.624423 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:27.624449 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:27.624465 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:27.624498 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0406 18:57:27.644182 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.646089 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.646152 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.646187 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.683275 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.755053 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.756580 37873 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0406 18:57:27.756730 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.756780 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.756865 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.private-shared-ip.example.com" }] } I0406 18:57:27.756921 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.757016 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.757139 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.757219 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 18:57:27.757202 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.757194 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.757333 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.758451 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.758500 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.758558 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.758630 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.757351 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0406 18:57:27.758914 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0406 18:57:27.758827 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:27.759256 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.759319 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 18:57:27.761856 37873 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0406 18:57:27.761957 37873 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 18:57:27.761986 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 18:57:27.761993 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0406 18:57:27.762099 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0406 18:57:27.762104 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.762218 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:27.762210 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.762304 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 18:57:27.762286 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.762378 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.762349 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0406 18:57:27.762403 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.762436 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.762475 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.762515 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.762621 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0406 18:57:27.762654 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.762792 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.763020 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0406 18:57:27.763015 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0406 18:57:27.763302 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0406 18:57:27.763419 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0406 18:57:27.763499 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I0406 18:57:27.763561 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0406 18:57:27.763622 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:27.763650 37873 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0406 18:57:27.763660 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I0406 18:57:27.763701 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0406 18:57:27.763782 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0406 18:57:27.763847 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.763907 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:27.763935 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:27.763956 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.764007 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:27.764039 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.764072 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:27.764099 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:27.764118 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.764167 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:27.764191 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.764212 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:27.763945 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 18:57:27.764238 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] } I0406 18:57:27.764299 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0406 18:57:27.764371 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:27.764413 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.764447 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.764491 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.765064 37873 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0406 18:57:27.765128 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0406 18:57:27.765180 37873 api.go:53] PageSize not implemented I0406 18:57:27.765242 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 18:57:27.765280 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 18:57:27.765339 37873 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "Name", Value: "api.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0406 18:57:27.765283 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.765386 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 18:57:27.765450 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 18:57:27.765495 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } W0406 18:57:27.765542 37873 api.go:53] PageSize not implemented I0406 18:57:27.765477 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 18:57:27.765554 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 18:57:27.765679 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 18:57:27.765665 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.765704 37873 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0406 18:57:27.765741 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 18:57:27.765738 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.765759 37873 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 18:57:27.765787 37873 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 18:57:27.765799 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.765899 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0406 18:57:27.765931 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 18:57:27.766133 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" }] }] } I0406 18:57:27.766280 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:27.766366 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.766447 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.766505 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0406 18:57:27.766558 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.766638 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.766725 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.766771 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:27.766844 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0406 18:57:27.766880 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.766971 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.767047 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.767130 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0406 18:57:27.767208 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0406 18:57:27.767285 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.767386 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0406 18:57:27.767467 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.767524 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.767619 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.767735 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.767810 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.767877 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:27.767915 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:27.767941 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0406 18:57:27.768008 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.768086 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.768163 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0406 18:57:27.768227 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.768327 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 18:57:27.768406 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0406 18:57:27.768439 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.768516 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0406 18:57:27.768622 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.768653 37873 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0406 18:57:27.768716 37873 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0406 18:57:27.768763 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.768811 37873 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:27.768814 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.768893 37873 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0406 18:57:27.768912 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.768949 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:27.768973 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:27.768997 37873 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0406 18:57:27.769030 37873 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0406 18:57:27.769090 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.769193 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0406 18:57:27.769237 37873 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 18:57:27.769263 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.769292 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.769324 37873 dns.go:238] Pre-creating DNS records I0406 18:57:27.769341 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.773107 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0406 18:57:27.774976 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.775027 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.775063 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.812760 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.892672 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.894751 37873 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0406 18:57:27.894948 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] }] } I0406 18:57:27.895185 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0406 18:57:27.895269 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0406 18:57:27.895305 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0406 18:57:27.895284 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:27.895404 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.895467 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:27.895575 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.896967 37873 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0406 18:57:27.897077 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0406 18:57:27.897243 37873 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 18:57:27.897268 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0406 18:57:27.897232 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.897305 37873 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 18:57:27.897321 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0406 18:57:27.897441 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.897480 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.897507 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0406 18:57:27.897607 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.897636 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0406 18:57:27.897720 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.897750 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0406 18:57:27.897831 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.897877 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.897963 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.898049 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0406 18:57:27.898072 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.898143 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:27.898166 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0406 18:57:27.899832 37873 executor.go:111] Tasks: 71 done / 116 total; 36 can run W0406 18:57:27.900018 37873 api.go:53] PageSize not implemented I0406 18:57:27.900031 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.900046 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 18:57:27.900099 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.900140 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0406 18:57:27.900202 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.900260 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0406 18:57:27.900300 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0406 18:57:27.900324 37873 api.go:53] PageSize not implemented I0406 18:57:27.900341 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 18:57:27.900399 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0406 18:57:27.900435 37873 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 18:57:27.900399 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.900490 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.900628 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.900719 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:27.900770 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.900819 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0406 18:57:27.900867 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.901025 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0406 18:57:27.901618 37873 executor.go:111] Tasks: 107 done / 116 total; 5 can run W0406 18:57:27.901709 37873 api.go:53] PageSize not implemented I0406 18:57:27.901680 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.901746 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:27.901725 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0406 18:57:27.901733 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 18:57:27.901791 37873 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0406 18:57:27.901817 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0406 18:57:27.901868 37873 api.go:53] PageSize not implemented I0406 18:57:27.901880 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 18:57:27.902002 37873 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0406 18:57:27.902118 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0406 18:57:27.902210 37873 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0406 18:57:27.904242 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:27.904285 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:27.904309 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:27.904332 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:27.904351 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:27.904372 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:27.904392 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:27.904411 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:27.904432 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:27.904456 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:27.904481 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0406 18:57:27.904503 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0406 18:57:27.904524 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:27.904566 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:27.904586 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:27.904607 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:27.904626 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0406 18:57:27.904646 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:27.904676 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:27.904700 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0406 18:57:27.904724 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0406 18:57:27.904745 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0406 18:57:27.904767 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:27.904793 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:27.904812 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0406 18:57:27.904835 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:27.904854 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:27.904873 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:27.904896 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:27.904918 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:27.905811 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:27.905829 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:27.905854 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.905891 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0406 18:57:27.905935 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.905969 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.905995 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.906018 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.906042 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.906068 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0406 18:57:27.906099 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.906130 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:27.906149 37873 natgateway.go:117] DescribeNatGateways: { } I0406 18:57:27.906173 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0406 18:57:27.906195 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0406 18:57:27.906221 37873 api.go:53] PageSize not implemented I0406 18:57:27.906232 37873 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0406 18:57:27.906263 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:27.906280 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:27.906291 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:27.906307 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:27.906319 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:27.906348 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:27.906366 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:27.906392 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:27.906417 37873 iamrole.go:89] ListRoles: { } I0406 18:57:27.906451 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:27.906478 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:27.906498 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0406 18:57:27.906519 37873 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0406 18:57:27.906538 37873 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:bastion.private-shared-ip.example.com ok autoscaling-group:nodes.private-shared-ip.example.com ok autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0406 18:57:27.906893 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0406 18:57:27.906897 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } load-balancer:bastion-private-shared-ip-eepmph ok I0406 18:57:27.906939 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.906999 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.907021 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 18:57:27.907049 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0406 18:57:27.906957 37873 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } nat-gateway:nat-1 ok I0406 18:57:27.907027 37873 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } autoscaling-config:lt-2 ok load-balancer:api-private-shared-ip-exa-ohatqj ok I0406 18:57:27.907095 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } volume:vol-2 ok iam-instance-profile:nodes.private-shared-ip.example.com ok keypair:key-1 ok I0406 18:57:27.907128 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 18:57:27.907032 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0406 18:57:27.907154 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:27.907156 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.907192 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0406 18:57:27.907185 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } W0406 18:57:27.907247 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.907257 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-3 ok autoscaling-config:lt-1 ok I0406 18:57:27.907303 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0406 18:57:27.907334 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0406 18:57:27.907403 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.907413 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0406 18:57:27.907431 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0406 18:57:27.907459 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0406 18:57:27.907569 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.907578 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0406 18:57:27.907628 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:27.907642 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.907692 37873 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0406 18:57:27.907656 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0406 18:57:27.907710 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.907703 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0406 18:57:27.907718 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:27.907729 37873 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0406 18:57:27.907743 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:27.907756 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.907787 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0406 18:57:27.907809 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0406 18:57:27.907812 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0406 18:57:27.907818 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0406 18:57:27.907890 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0406 18:57:27.907851 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:27.907929 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:27.907948 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0406 18:57:27.907913 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0406 18:57:27.907988 37873 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0406 18:57:27.907999 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0406 18:57:27.908019 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0406 18:57:27.908072 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:27.908105 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0406 18:57:27.908294 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.28s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0406 18:57:27.908541 37873 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:27.908743 37873 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:27.908772 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:27.908796 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:27.908827 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:27.908852 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:27.908871 37873 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:27.908893 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:27.908915 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:27.908953 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0406 18:57:27.929254 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0406 18:57:27.932524 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.932609 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:27.970120 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:28.060051 37873 iamrole.go:89] ListRoles: { } I0406 18:57:28.061228 37873 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0406 18:57:28.063056 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.063190 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:28.063336 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:28.063534 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:28.063600 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.063837 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:28.063821 37873 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.063917 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.064217 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:28.064438 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0406 18:57:28.064407 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0406 18:57:28.064488 37873 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0406 18:57:28.064859 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 18:57:28.065046 37873 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.065126 37873 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0406 18:57:28.064558 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.065907 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.065944 37873 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.066112 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.066276 37873 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0406 18:57:28.066384 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:28.066420 37873 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" }] }] } I0406 18:57:28.066491 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:28.067059 37873 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0406 18:57:28.067136 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.067188 37873 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 18:57:28.067270 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 18:57:28.067273 37873 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:28.067303 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 18:57:28.067317 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.067393 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0406 18:57:28.067468 37873 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0406 18:57:28.067510 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.067602 37873 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.067648 37873 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.067705 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:28.067736 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0406 18:57:28.067886 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0406 18:57:28.067915 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 18:57:28.068058 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:28.068127 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:28.068158 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:28.068183 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.068294 37873 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0406 18:57:28.068326 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.068444 37873 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" } ] }], VpcId: "vpc-1" } I0406 18:57:28.068502 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:28.068553 37873 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0406 18:57:28.068610 37873 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0406 18:57:28.068670 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:28.068709 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:28.068748 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.068783 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.068605 37873 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.070270 37873 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0406 18:57:28.070415 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.070534 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.070672 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.070855 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.070925 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.070991 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.071104 37873 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:28.071189 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0406 18:57:28.071300 37873 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0406 18:57:28.071333 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" }] }] } I0406 18:57:28.071420 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" }] }] } I0406 18:57:28.071552 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.071682 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.071774 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.071843 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0406 18:57:28.071883 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.071985 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.072080 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0406 18:57:28.072236 37873 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.072423 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.072637 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.072815 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.073006 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.073136 37873 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0406 18:57:28.073176 37873 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0406 18:57:28.074198 37873 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0406 18:57:28.074348 37873 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0406 18:57:28.074498 37873 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0406 18:57:28.074535 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:28.074570 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:28.074606 37873 dns.go:238] Pre-creating DNS records I0406 18:57:28.074626 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:28.078256 37873 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0406 18:57:28.080075 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.080129 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.116577 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:28.202126 37873 iamrole.go:89] ListRoles: { } I0406 18:57:28.203235 37873 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0406 18:57:28.203513 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.203660 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.203776 37873 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0406 18:57:28.203850 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.203883 37873 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.203919 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.204039 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.204135 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:28.204191 37873 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0406 18:57:28.204244 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.206512 37873 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0406 18:57:28.206814 37873 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 18:57:28.207076 37873 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 18:57:28.206989 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.207294 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.207483 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.207947 37873 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.207996 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0406 18:57:28.208166 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:28.208214 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0406 18:57:28.208335 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.208672 37873 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.208745 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:28.211228 37873 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0406 18:57:28.211331 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.211421 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.211536 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.211678 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.211815 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.211980 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.212061 37873 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0406 18:57:28.212702 37873 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0406 18:57:28.212840 37873 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0406 18:57:28.212973 37873 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0406 18:57:28.214695 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0406 18:57:28.214734 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0406 18:57:28.214754 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0406 18:57:28.214772 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0406 18:57:28.214792 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0406 18:57:28.214813 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0406 18:57:28.214832 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0406 18:57:28.214849 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0406 18:57:28.214869 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0406 18:57:28.214895 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0406 18:57:28.214918 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0406 18:57:28.214939 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0406 18:57:28.214961 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0406 18:57:28.214980 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0406 18:57:28.214999 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0406 18:57:28.215018 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0406 18:57:28.215037 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:28.215057 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0406 18:57:28.215077 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.215100 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0406 18:57:28.215119 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0406 18:57:28.215139 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0406 18:57:28.215158 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0406 18:57:28.215178 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0406 18:57:28.215199 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.215219 37873 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0406 18:57:28.216082 37873 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0406 18:57:28.216101 37873 keypairs.go:95] DescribeKeyPairs: { } I0406 18:57:28.216125 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216156 37873 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216189 37873 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216225 37873 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216254 37873 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216286 37873 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216311 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216334 37873 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216363 37873 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216393 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0406 18:57:28.216417 37873 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0406 18:57:28.216450 37873 api.go:53] PageSize not implemented I0406 18:57:28.216464 37873 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0406 18:57:28.216480 37873 loadbalancers.go:34] PageSize not implemented I0406 18:57:28.216492 37873 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0406 18:57:28.216506 37873 targetgroups.go:35] PageSize not implemented I0406 18:57:28.216517 37873 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0406 18:57:28.216548 37873 zones.go:93] ListHostedZonesPages { } I0406 18:57:28.216568 37873 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0406 18:57:28.216598 37873 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0406 18:57:28.216632 37873 iamrole.go:89] ListRoles: { } I0406 18:57:28.216661 37873 internetgateways.go:101] DescribeInternetGateways: { } I0406 18:57:28.216685 37873 routetable.go:53] DescribeRouteTables: { } I0406 18:57:28.216705 37873 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0406 18:57:28.216972 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } autoscaling-config:lt-2 ok I0406 18:57:28.217015 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0406 18:57:28.217025 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0406 18:57:28.217074 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0406 18:57:28.217082 37873 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.217132 37873 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok I0406 18:57:28.217117 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0406 18:57:28.217257 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:28.217275 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0406 18:57:28.217172 37873 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.nthsqsresources.longclustername.example.com. ok I0406 18:57:28.217296 37873 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0406 18:57:28.217323 37873 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0406 18:57:28.217390 37873 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0406 18:57:28.217398 37873 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0406 18:57:28.217416 37873 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0406 18:57:28.217444 37873 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0406 18:57:28.217476 37873 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0406 18:57:28.217508 37873 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0406 18:57:28.217562 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.217564 37873 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0406 18:57:28.217593 37873 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0406 18:57:28.217601 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0406 18:57:28.217623 37873 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0406 18:57:28.217652 37873 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 18:57:28.217703 37873 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0406 18:57:28.217713 37873 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0406 18:57:28.217733 37873 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0406 18:57:28.217782 37873 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0406 18:57:28.217833 37873 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0406 18:57:28.217894 37873 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0406 18:57:28.218079 37873 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.31s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 50.595s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.005s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0406 18:56:41.306336 38385 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.004s === RUN TestPodController I0406 18:56:41.329611 38388 pod.go:59] starting pod controller I0406 18:56:41.329927 38388 stoppable.go:62] shutting down controller I0406 18:56:41.330009 38388 pod.go:107] Got stop signal --- PASS: TestPodController (0.00s) PASS I0406 18:56:41.330076 38388 pod.go:65] shutting down pod controller ok k8s.io/kops/dns-controller/pkg/watchers 0.029s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.004s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0000a5f40} route53_test.go:93: Got zone list: [0xc0000a5f70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc0000120c0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003600a0} route53_test.go:93: Got zone list: [0xc0003600d0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc0000120d8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc0003601d0} route53_test.go:170: Successfully added managed DNS zone: &{0xc0003318f0 0xc0000120f8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360390} route53_test.go:93: Got zone list: [0xc0003603c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012118} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360520} route53_test.go:93: Got zone list: [0xc000360550] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012140} route53_test.go:187: Successfully added resource record set: {0xc000180600 0xc000012160} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360830} route53_test.go:93: Got zone list: [0xc000360860] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc0000121f0} route53_test.go:199: Successfully added resource record set: {0xc000180780 0xc000012210} route53_test.go:118: Got 1 recordsets: [0xc000360a00] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360b90} route53_test.go:93: Got zone list: [0xc000360bc0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc0000122b0} route53_test.go:221: Successfully added resource record set: {0xc000180900 0xc0000122d0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000180900 0xc0000122d0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360f80} route53_test.go:93: Got zone list: [0xc000360fb0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012390} route53_test.go:246: Successfully removed resource set {0xc000180b00 0xc0000123b0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361290} route53_test.go:93: Got zone list: [0xc0003612c0] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012440} route53_test.go:264: Successfully removed resource set {0xc000180c80 0xc000012460} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003615d0} route53_test.go:93: Got zone list: [0xc000361600] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc0000124f8} commontests.go:192: Successfully added resource record set: {0xc000180e00 0xc000012518} commontests.go:45: Correctly replaced resource record {0xc000180e00 0xc000012518} -> {0xc000180f00 0xc000012568} commontests.go:142: Got 1 recordsets: [0xc000361910] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361b40} route53_test.go:93: Got zone list: [0xc000361b70] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012620} commontests.go:192: Successfully added resource record set: {0xc000181180 0xc000012640} commontests.go:70: Correctly replaced resource record {0xc000181180 0xc000012640} -> {0xc000181280 0xc000012690} commontests.go:142: Got 1 recordsets: [0xc000361e80] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036e100} route53_test.go:93: Got zone list: [0xc00036e130] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012758} commontests.go:192: Successfully added resource record set: {0xc000181500 0xc000012778} commontests.go:142: Got 2 recordsets: [0xc00036e3a0 0xc00036e3c0] commontests.go:142: Got 2 recordsets: [0xc00036e430 0xc00036e450] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00036e650} route53_test.go:93: Got zone list: [0xc00036e680] route53_test.go:98: Got at least 1 zone in list:&{0xc0003316b0 0xc000012880} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.011s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.006s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520440} clouddns_test.go:71: Got zone list: [0xc000010750] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc0000106f0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520540} clouddns_test.go:71: Got zone list: [0xc0000108d0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000010870} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520640} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00012a340 0xc000010a50} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520700} clouddns_test.go:71: Got zone list: [0xc000010bd0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000010b70} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520820} clouddns_test.go:71: Got zone list: [0xc000010e10] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000010d68} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520960} clouddns_test.go:71: Got zone list: [0xc0000110c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011098} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520ac0} clouddns_test.go:71: Got zone list: [0xc000011350] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011320} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520c20} clouddns_test.go:71: Got zone list: [0xc0000114e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc0000114b8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520d60} clouddns_test.go:71: Got zone list: [0xc0000116b0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011680} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000520ec0} clouddns_test.go:71: Got zone list: [0xc000011878] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011848} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000521080} clouddns_test.go:71: Got zone list: [0xc000011b48] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011b18} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc000521200} clouddns_test.go:71: Got zone list: [0xc000011e78] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc000011e48} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000013588 0xc0005213e0} clouddns_test.go:71: Got zone list: [0xc0001342d0] clouddns_test.go:76: Got at least 1 zone in list:&{0xc000481f80 0xc0001342a0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.056s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.024s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.069s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0406 18:56:49.949445 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.949695 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.949745 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.949782 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.949838 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.949870 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.949888 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.949918 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.949958 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:49.959903 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.01s) === RUN TestContainerdBuilder_Docker_19_03_14 I0406 18:56:49.964211 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.964449 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.964501 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.964564 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.964624 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.964704 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.964745 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.964786 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.964813 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:49.966004 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0406 18:56:49.970193 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.970366 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.970447 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.970526 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.970610 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.970687 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.970751 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.970827 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.970888 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:49.973022 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:49.975776 38927 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0406 18:56:49.975812 38927 assetstore.go:111] bin/containerd I0406 18:56:49.975832 38927 assetstore.go:111] bin/containerd-shim I0406 18:56:49.975847 38927 assetstore.go:111] bin/containerd-shim-runc-v1 I0406 18:56:49.975861 38927 assetstore.go:111] bin/containerd-shim-runc-v2 I0406 18:56:49.975875 38927 assetstore.go:111] bin/containerd-stress I0406 18:56:49.975889 38927 assetstore.go:111] bin/ctr I0406 18:56:49.975960 38927 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0406 18:56:49.975989 38927 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0406 18:56:49.981508 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.981779 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.981838 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.981893 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.981955 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.982004 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.982046 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.982119 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.982155 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:49.984097 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:49.986660 38927 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.01s) === RUN TestContainerdBuilder_SkipInstall I0406 18:56:49.988406 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.988669 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.988720 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.988771 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.988827 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.988948 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.988982 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.989023 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.989056 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:49.990805 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:49.997306 38927 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.01s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0406 18:56:49.998620 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:49.998908 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:49.998971 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:49.999023 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:49.999076 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:49.999171 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:49.999204 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:49.999241 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:49.999270 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.000898 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:50.002945 38927 assetstore.go:108] Matching assets for "^docker/": I0406 18:56:50.002980 38927 assetstore.go:111] docker/containerd I0406 18:56:50.003001 38927 assetstore.go:111] docker/containerd-shim I0406 18:56:50.003017 38927 assetstore.go:111] docker/ctr I0406 18:56:50.003032 38927 assetstore.go:111] docker/docker I0406 18:56:50.003047 38927 assetstore.go:111] docker/docker-init I0406 18:56:50.003062 38927 assetstore.go:111] docker/docker-proxy I0406 18:56:50.003079 38927 assetstore.go:111] docker/dockerd I0406 18:56:50.003096 38927 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0406 18:56:50.007276 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.007490 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.007553 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.007597 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.007654 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.007736 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.007781 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.007827 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.007855 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.009382 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:50.011421 38927 assetstore.go:108] Matching assets for "^docker/": I0406 18:56:50.011446 38927 assetstore.go:111] docker/containerd I0406 18:56:50.011461 38927 assetstore.go:111] docker/containerd-shim I0406 18:56:50.011474 38927 assetstore.go:111] docker/ctr I0406 18:56:50.011524 38927 assetstore.go:111] docker/docker I0406 18:56:50.011540 38927 assetstore.go:111] docker/docker-init I0406 18:56:50.011554 38927 assetstore.go:111] docker/docker-proxy I0406 18:56:50.011567 38927 assetstore.go:111] docker/dockerd I0406 18:56:50.011582 38927 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0406 18:56:50.015857 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.016084 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.016142 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.016194 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.016247 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.016325 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.016365 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.016410 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.016441 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.017823 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:50.020223 38927 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0406 18:56:50.020550 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.020818 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.020872 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.020925 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.020985 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.021069 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.021102 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.021158 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.021189 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.023068 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:50.026008 38927 assetstore.go:108] Matching assets for "^docker/": I0406 18:56:50.026041 38927 assetstore.go:111] docker/docker I0406 18:56:50.026058 38927 assetstore.go:111] docker/docker-containerd I0406 18:56:50.026073 38927 assetstore.go:111] docker/docker-containerd-ctr I0406 18:56:50.026104 38927 assetstore.go:111] docker/docker-containerd-shim I0406 18:56:50.026121 38927 assetstore.go:111] docker/docker-init I0406 18:56:50.026136 38927 assetstore.go:111] docker/docker-proxy I0406 18:56:50.026169 38927 assetstore.go:111] docker/docker-runc I0406 18:56:50.026189 38927 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0406 18:56:50.034457 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.034787 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.034881 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.034938 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.034987 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.035063 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.035120 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.035184 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.035234 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.038597 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.01s) === RUN TestDockerHooksBuilder I0406 18:56:50.041437 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.041722 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.041799 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.041904 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.041983 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.042049 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.042105 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.042166 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.042216 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.045681 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.01s) === RUN TestKopsControllerBuilder I0406 18:56:50.050536 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.050818 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.050875 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.050930 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.050990 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.051037 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.051071 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.051112 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.051141 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.054512 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.01s) === RUN TestKubeAPIServerBuilder I0406 18:56:50.068657 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.068953 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.069012 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.069115 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.069178 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.069229 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.069269 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.069335 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.069375 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.072776 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.02s) === RUN TestDeddicatedAPIServerBuilder I0406 18:56:50.088922 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.089163 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.089217 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.089267 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.089322 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.089370 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.089407 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.089452 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.089482 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.092621 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.02s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0406 18:56:50.111461 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.111708 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.111763 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.111814 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.111869 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.111914 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.111951 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.111997 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.112029 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.114851 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.03s) === RUN TestAwsIamAuthenticator I0406 18:56:50.139239 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.139531 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.139618 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.139707 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.139768 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.139815 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.139851 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.139892 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.139925 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.142686 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.02s) === RUN TestKubeAPIServerBuilderAMD64 I0406 18:56:50.155994 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:50.156294 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:50.156354 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:50.156407 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:50.156465 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:50.156512 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:50.156547 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:50.156593 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:50.156624 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:50.159829 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.57s) === RUN TestKubeAPIServerBuilderARM64 I0406 18:56:51.721792 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.722108 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.722160 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.722208 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.722262 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.722311 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.722341 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.722378 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.722409 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.725651 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.08s) === RUN TestKubeControllerManagerBuilder I0406 18:56:51.802027 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.802235 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.802273 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.802304 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.802336 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.802365 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.802385 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.802409 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.802428 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.804367 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.01s) === RUN TestKubeControllerManagerBuilderAMD64 I0406 18:56:51.811267 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.811464 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.811504 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.811534 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.811567 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.811599 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.811617 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.811642 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.811661 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.813558 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.09s) === RUN TestKubeControllerManagerBuilderARM64 I0406 18:56:51.897333 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.897615 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.897671 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.897727 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.897784 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.897833 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.897881 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.897928 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.897959 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.901327 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.08s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0406 18:56:51.976147 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.976349 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.976387 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.976421 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.976455 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.976483 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.976501 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.976524 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.976542 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.978452 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0406 18:56:51.982236 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:51.982396 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:51.982427 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:51.982454 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:51.982481 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:51.982504 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:51.982522 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:51.982544 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:51.982559 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:51.984227 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.07s) === RUN TestKubeProxyBuilderARM64 I0406 18:56:52.054870 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.055127 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.055207 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.055283 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.055361 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.055427 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.055485 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.055589 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.055621 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.057773 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.07s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0406 18:56:52.126784 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.127017 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.127065 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.127106 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.127151 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.127189 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.127218 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.127260 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.127285 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.130071 38927 zones.go:93] ListHostedZonesPages { } W0406 18:56:52.134076 38927 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0406 18:56:52.137506 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.137820 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.137951 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.138443 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.138534 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.138590 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.138625 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.138671 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.138702 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.141835 38927 zones.go:93] ListHostedZonesPages { } W0406 18:56:52.206220 38927 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.07s) === RUN TestKubeSchedulerBuilderARM64 I0406 18:56:52.209243 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.209476 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.209527 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.209579 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.209630 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.209670 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.209701 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.209739 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.209766 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.212817 38927 zones.go:93] ListHostedZonesPages { } W0406 18:56:52.278956 38927 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.07s) === RUN TestKubectlBuilder I0406 18:56:52.282071 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.282328 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.282383 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.282439 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.282497 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.282548 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.282585 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.282628 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.282660 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.285839 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:52.289142 38927 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0406 18:56:52.292586 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.292829 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.292908 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.292959 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.293015 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.293060 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.293098 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.293141 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.293172 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.295585 38927 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0406 18:56:52.301652 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.301882 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.301959 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.302013 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.302071 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.302117 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.302153 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.302200 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.302230 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.304622 38927 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0406 18:56:52.310496 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.310752 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.310808 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.310855 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.310911 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.310957 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.310995 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.311279 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.311559 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.314602 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:52.316285 38927 assetstore.go:108] Matching assets for "protokube$": I0406 18:56:52.316303 38927 assetstore.go:111] /opt/kops/bin/protokube I0406 18:56:52.316325 38927 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0406 18:56:52.316348 38927 assetstore.go:108] Matching assets for "channels$": I0406 18:56:52.316359 38927 assetstore.go:111] /opt/kops/bin/channels I0406 18:56:52.316366 38927 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0406 18:56:52.317806 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.318004 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.318038 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.318070 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.318101 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.318127 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.318145 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.318168 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.318183 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.319794 38927 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0406 18:56:52.321981 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.322111 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.322135 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.322170 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.322200 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.322221 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.322239 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.322263 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.322277 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.323835 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:52.325403 38927 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0406 18:56:52.325813 38927 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:56:52.325951 38927 internetgateways.go:66] CreateInternetGateway: { } I0406 18:56:52.325976 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:56:52.326000 38927 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:56:52.326028 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:56:52.326050 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:56:52.326069 38927 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:56:52.326091 38927 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:56:52.326105 38927 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:56:52.327668 38927 zones.go:93] ListHostedZonesPages { } I0406 18:56:52.329257 38927 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.470s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.028s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.01s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.052s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.031s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.031s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0406 18:56:49.876391 38933 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0406 18:56:49.876527 38933 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.014s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0406 18:56:49.945501 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.945604 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.945650 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.945698 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.945736 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0406 18:56:49.946048 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946095 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946125 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946156 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946190 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946217 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946252 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946281 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946316 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946352 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946385 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946425 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946460 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946495 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946527 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946599 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946636 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946678 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0406 18:56:49.946711 38934 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0406 18:56:49.946903 38934 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0406 18:56:49.946978 38934 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.095s === RUN TestApplySet I0406 18:56:49.898677 38936 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0406 18:56:49.899652 38936 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0406 18:56:49.900584 38936 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0406 18:56:49.901095 38936 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0406 18:56:49.902478 38936 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0406 18:56:49.902800 38936 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0406 18:56:49.903274 38936 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0406 18:56:49.903440 38936 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0406 18:56:49.909987 38936 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I0406 18:56:49.910113 38936 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I0406 18:56:49.910549 38936 health.go:33] status conditions not found for /v1, Kind=Namespace I0406 18:56:49.910820 38936 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I0406 18:56:49.910886 38936 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I0406 18:56:49.911150 38936 health.go:33] status conditions not found for /v1, Kind=ConfigMap I0406 18:56:49.911451 38936 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I0406 18:56:49.911525 38936 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I0406 18:56:49.912240 38936 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I0406 18:56:49.912302 38936 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.02s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.049s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0406 18:56:50.003555 38929 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.10s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0406 18:56:50.004883 38929 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0406 18:56:50.006085 38929 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0406 18:56:50.007169 38929 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0406 18:56:50.008406 38929 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0406 18:56:50.009487 38929 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0406 18:56:50.010568 38929 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0406 18:56:50.011394 38929 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0406 18:56:50.011416 38929 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0406 18:56:50.011427 38929 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0406 18:56:50.011438 38929 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.148s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.015s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.059s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.022s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.081s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.024s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.004s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.003s === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.037s === RUN TestFlagToFalse I0406 18:56:57.409878 40353 featureflag_test.go:32] Created flag Unittest1 I0406 18:56:57.409925 40353 featureflag.go:160] FeatureFlag "UnitTest1"=false I0406 18:56:57.409935 40353 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0406 18:56:57.410002 40353 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.005s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.036s === RUN TestRenderInstanceGroupZones W0406 18:56:57.426975 40354 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.023s === RUN TestWarmPoolOnlyRoll I0406 18:57:09.898524 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.898632 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.898651 40935 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0406 18:57:09.899231 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:09.899276 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.900403 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.900430 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.905603 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.905652 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.905667 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.905678 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:09.907239 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:09.907792 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:09.907821 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.908945 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.908969 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.914187 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.914256 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:09.915239 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:09.915280 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.916423 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.916444 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.921641 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.921685 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.921707 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.921723 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:09.922736 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:09.923489 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:09.923587 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.924769 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.924814 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.930026 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.930086 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:09.930968 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:09.931002 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.932157 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.932205 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.937387 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.937456 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:09.938517 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:09.938557 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.939706 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.939747 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.944958 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.944994 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.945017 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.945036 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:09.946101 40935 instancegroups.go:407] Draining the node: "node-2a.local". I0406 18:57:09.946951 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:09.946987 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.948138 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.948180 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.953408 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.953492 40935 instancegroups.go:407] Draining the node: "node-2b.local". I0406 18:57:09.954432 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:09.954470 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.955642 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.955698 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.960938 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.961009 40935 instancegroups.go:407] Draining the node: "node-2c.local". I0406 18:57:09.961943 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:09.961980 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.963130 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.963153 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:09.968358 40935 instancegroups.go:510] Cluster validated. I0406 18:57:09.968388 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.07s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0406 18:57:09.968762 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:09.968784 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.969924 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.969955 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.969978 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.969988 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:09.970003 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.971095 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.971125 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.971134 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:09.971150 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.972261 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.972289 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.972332 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.972350 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:09.972374 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.973517 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.973548 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.973557 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:09.973578 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.974690 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.974731 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.974747 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:09.974770 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.975903 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.975919 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.975945 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.975955 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:09.975971 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.977078 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.977113 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.977127 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:09.977150 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.978285 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0406 18:57:09.978317 40935 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0406 18:57:09.978327 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:09.978344 40935 instancegroups.go:443] waiting for 1ms after terminating instance W0406 18:57:09.979454 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0406 18:57:09.979479 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0406 18:57:09.979792 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:09.979820 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.981444 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.981481 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.981502 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.981547 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.981573 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.981591 40935 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.981606 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:09.982292 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:09.983069 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:09.983103 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.984229 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.984259 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.984276 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.984311 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:09.985130 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:09.985164 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.986282 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.986317 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.986336 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.986356 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.986376 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.986392 40935 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.986412 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:09.987468 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:09.988290 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:09.988324 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.989442 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.989476 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.989494 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.989528 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:09.990260 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:09.990297 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.991600 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.991633 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.991661 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.991701 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:09.992217 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:09.992245 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.993385 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.993408 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.993421 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.993430 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.993441 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.993449 40935 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.993458 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:09.994169 40935 instancegroups.go:407] Draining the node: "node-2a.local". I0406 18:57:09.994659 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:09.994682 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.995800 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.995832 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.995852 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.995889 40935 instancegroups.go:407] Draining the node: "node-2b.local". I0406 18:57:09.996662 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:09.996692 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.997798 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.997820 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.997848 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.997886 40935 instancegroups.go:407] Draining the node: "node-2c.local". I0406 18:57:09.998368 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:09.998389 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:09.999512 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:09.999545 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0406 18:57:09.999562 40935 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0406 18:57:09.999581 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0406 18:57:09.999846 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0406 18:57:10.000090 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.000121 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.001250 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.001271 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.006450 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.006508 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.006524 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.006533 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.007090 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.007660 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.007692 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.014551 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.014609 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.019832 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.019892 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:10.020626 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:10.020661 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.021790 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.021822 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.027042 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.027081 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.027096 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.027107 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.027831 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.028304 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.028330 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.029460 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.029501 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.034718 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.034781 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.035688 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.035727 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.036867 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.036897 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.042109 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.042182 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.043085 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.043121 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.045131 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.045167 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.050362 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.050400 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.050429 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.050448 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:10.051868 40935 instancegroups.go:407] Draining the node: "node-2a.local". I0406 18:57:10.052335 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:10.052370 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.053502 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.053531 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.058669 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.058735 40935 instancegroups.go:407] Draining the node: "node-2b.local". I0406 18:57:10.059309 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:10.059335 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.060450 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.060471 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.065658 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.065718 40935 instancegroups.go:407] Draining the node: "node-2c.local". I0406 18:57:10.066351 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:10.066386 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.067506 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.067528 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.072733 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.072777 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0406 18:57:10.072929 40935 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0406 18:57:10.073295 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.073317 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.074437 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.074471 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.074491 40935 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0406 18:57:10.074819 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.074848 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.075972 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.075995 40935 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0406 18:57:10.076009 40935 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0406 18:57:10.076156 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.076174 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.076183 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.076193 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0406 18:57:10.076202 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0406 18:57:10.076280 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.076297 40935 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0406 18:57:10.076306 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.076315 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0406 18:57:10.076324 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0406 18:57:10.076440 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.076461 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.077590 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.077623 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.082833 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.082919 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.082948 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.082966 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.083729 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.084509 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.084560 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.085686 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.085725 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.085747 40935 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0406 18:57:10.086061 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.086091 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.087224 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.087257 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.092497 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.092581 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.092608 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.092623 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.093382 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.094060 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.094095 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.095228 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.095270 40935 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0406 18:57:10.095292 40935 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0406 18:57:10.095595 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.095619 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.095634 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.096616 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.097348 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.097461 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.098617 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.098650 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.103862 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.103932 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.104893 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.104931 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.106048 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.106072 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.111302 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.111365 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.111960 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.111989 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.113111 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.113138 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.118354 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.118402 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0406 18:57:10.118770 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.118805 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.118833 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.118856 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0406 18:57:10.118878 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0406 18:57:10.119149 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.119180 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.119198 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.119224 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0406 18:57:10.119247 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0406 18:57:10.119517 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.119548 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.119569 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.119587 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0406 18:57:10.119606 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0406 18:57:10.119803 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.119835 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.119855 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.121025 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.121846 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.121882 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.123010 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.123050 40935 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0406 18:57:10.123076 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.123099 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0406 18:57:10.123118 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0406 18:57:10.123321 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.123348 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.123368 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.124326 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.125030 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.125063 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.126180 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.126211 40935 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0406 18:57:10.126230 40935 instancegroups.go:482] Cluster did not validate within 0s E0406 18:57:10.126250 40935 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0406 18:57:10.126271 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0406 18:57:10.126522 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.126552 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.127677 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.127718 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.128838 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.134095 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.135249 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.140459 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.141626 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.146874 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.146942 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.146959 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.146971 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.147552 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.148232 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.148267 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.149400 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.149440 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.154644 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.154712 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:10.155624 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:10.155663 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.156782 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.156838 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.162058 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.162105 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.162136 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.162154 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.165181 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.165998 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.166029 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.167165 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.167224 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.172462 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.172549 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.173349 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.173383 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.174505 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.174555 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.179715 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.179779 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.180405 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.180430 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.181542 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.181578 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.186796 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.186829 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.186847 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.186859 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:10.187516 40935 instancegroups.go:407] Draining the node: "node-2a.local". I0406 18:57:10.188093 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:10.188128 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.189262 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.189316 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.194557 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.194624 40935 instancegroups.go:407] Draining the node: "node-2b.local". I0406 18:57:10.195467 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:10.195500 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.196649 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.196684 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.202104 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.202169 40935 instancegroups.go:407] Draining the node: "node-2c.local". I0406 18:57:10.202766 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:10.202792 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.203927 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.203958 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.209162 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.209200 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0406 18:57:10.209535 40935 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0406 18:57:10.209564 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.210718 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.210747 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.211854 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.212972 40935 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0406 18:57:10.214170 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.219368 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.219447 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.219474 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.219491 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.220321 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.221081 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.221117 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.222267 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.222292 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.227482 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.227545 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:10.228426 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:10.228459 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.229602 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.229628 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.234819 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.234866 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.234887 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.234905 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.235926 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.236666 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.236736 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.237887 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.237927 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.243105 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.243165 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.243956 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.243989 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.245125 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.245157 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.250355 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.250428 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.251506 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.251543 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.252666 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.252712 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.257917 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.257961 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.257986 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.258007 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:10.259212 40935 instancegroups.go:407] Draining the node: "node-2a.local". I0406 18:57:10.260136 40935 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0406 18:57:10.260176 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.261297 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.261331 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.266567 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.266680 40935 instancegroups.go:407] Draining the node: "node-2b.local". I0406 18:57:10.268575 40935 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0406 18:57:10.268662 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.269824 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.269861 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.275064 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.275121 40935 instancegroups.go:407] Draining the node: "node-2c.local". I0406 18:57:10.275747 40935 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0406 18:57:10.275776 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.276891 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.276916 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.282115 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.282152 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0406 18:57:10.282587 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.282604 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.282614 40935 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0406 18:57:10.283109 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.283574 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.283600 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.284711 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.284735 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.290247 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.290302 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.290893 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.290923 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.292037 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.292077 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.297315 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.297348 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.01s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0406 18:57:10.297560 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.297579 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.297587 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.298095 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.298598 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.298622 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.300562 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.300618 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.305839 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.305903 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:10.306572 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:10.306599 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.307727 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.307755 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.312956 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.312992 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0406 18:57:10.313307 40935 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W0406 18:57:10.313337 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0406 18:57:10.313350 40935 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W0406 18:57:10.313360 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0406 18:57:10.313371 40935 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W0406 18:57:10.313381 40935 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0406 18:57:10.313392 40935 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0406 18:57:10.313402 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0406 18:57:10.313549 40935 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I0406 18:57:10.313586 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:10.314707 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.314736 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.319963 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.319991 40935 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I0406 18:57:10.320033 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.320049 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.320060 40935 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0406 18:57:10.320582 40935 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I0406 18:57:10.320600 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.320611 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.320620 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:10.321160 40935 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0406 18:57:10.321216 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:10.322364 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.322387 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.327562 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.327593 40935 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I0406 18:57:10.327606 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.327621 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.327633 40935 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0406 18:57:10.328318 40935 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0406 18:57:10.328364 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:10.329497 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.329518 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:10.334709 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.334746 40935 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0406 18:57:10.334762 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0406 18:57:10.335037 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.335060 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.335071 40935 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0406 18:57:10.336448 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.337093 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.337123 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.338246 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.338283 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.338323 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.338369 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.339483 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.339749 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.339801 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.340940 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.340994 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.441364 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.441433 40935 instancegroups.go:407] Draining the node: "node-1e.local". I0406 18:57:10.441465 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:10.442484 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:10.442708 40935 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 18:57:10.442751 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.443876 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.443907 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.443938 40935 instancegroups.go:407] Draining the node: "node-1f.local". I0406 18:57:10.444655 40935 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 18:57:10.444692 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.445804 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.445852 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.546188 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.546244 40935 instancegroups.go:407] Draining the node: "node-1g.local". I0406 18:57:10.546833 40935 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0406 18:57:10.546868 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.547997 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.548043 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.548062 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0406 18:57:10.548300 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.548328 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.548342 40935 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0406 18:57:10.549963 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.550028 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.550887 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.551095 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.551129 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.552289 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.552333 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.652748 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.652824 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:10.652880 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:10.653824 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:10.653982 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:10.654009 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.655129 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.655163 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.655194 40935 instancegroups.go:407] Draining the node: "node-1e.local". I0406 18:57:10.655837 40935 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 18:57:10.655874 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.657017 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.657058 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.757475 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.757545 40935 instancegroups.go:407] Draining the node: "node-1f.local". I0406 18:57:10.758324 40935 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 18:57:10.758369 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.759506 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.759544 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.759563 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0406 18:57:10.759794 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.759825 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.759838 40935 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I0406 18:57:10.761677 40935 instancegroups.go:407] Draining the node: "master-1a.local". I0406 18:57:10.762338 40935 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0406 18:57:10.762381 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.763515 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.763556 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.763597 40935 instancegroups.go:407] Draining the node: "master-1c.local". I0406 18:57:10.763635 40935 instancegroups.go:407] Draining the node: "master-1b.local". I0406 18:57:10.764566 40935 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0406 18:57:10.764732 40935 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0406 18:57:10.764758 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.765871 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.765925 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.866274 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.866335 40935 instancegroups.go:407] Draining the node: "master-1e.local". I0406 18:57:10.866358 40935 instancegroups.go:407] Draining the node: "master-1d.local". I0406 18:57:10.867318 40935 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0406 18:57:10.867576 40935 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0406 18:57:10.867621 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.868737 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.868764 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.868786 40935 instancegroups.go:407] Draining the node: "master-1f.local". I0406 18:57:10.869242 40935 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0406 18:57:10.869268 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.870386 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.870438 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.970863 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.970939 40935 instancegroups.go:407] Draining the node: "master-1g.local". I0406 18:57:10.971615 40935 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0406 18:57:10.971653 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.972820 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.972866 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.972879 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0406 18:57:10.973416 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.973440 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.973449 40935 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0406 18:57:10.974506 40935 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0406 18:57:10.974564 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:10.975711 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.975745 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.975753 40935 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0406 18:57:10.975790 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:10.976931 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.976957 40935 instancegroups.go:510] Cluster validated. I0406 18:57:10.976985 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:10.977030 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:10.977513 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:10.977894 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:10.977926 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:10.979073 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:10.979129 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.079569 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.079638 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:11.079681 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:11.080600 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:11.080829 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:11.080867 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.082010 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.082039 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.082061 40935 instancegroups.go:407] Draining the node: "node-1e.local". I0406 18:57:11.082525 40935 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 18:57:11.082556 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.083662 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.083705 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.184138 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.184193 40935 instancegroups.go:407] Draining the node: "node-1f.local". I0406 18:57:11.184779 40935 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 18:57:11.184818 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.185938 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.185972 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.185994 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0406 18:57:11.186271 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.186302 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.186315 40935 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0406 18:57:11.187948 40935 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0406 18:57:11.187987 40935 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0406 18:57:11.188030 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:11.189175 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.189202 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.189226 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.189272 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.189697 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.190075 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.190105 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.191226 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.191273 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.291752 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.291836 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:11.291843 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:11.292898 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:11.293129 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:11.293170 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.294296 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.294329 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.294364 40935 instancegroups.go:407] Draining the node: "node-1e.local". I0406 18:57:11.295088 40935 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 18:57:11.295131 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.296244 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.296298 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.396823 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.396882 40935 instancegroups.go:407] Draining the node: "node-1f.local". I0406 18:57:11.398356 40935 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 18:57:11.398405 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.399548 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.399576 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.399589 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0406 18:57:11.399748 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.399763 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.399773 40935 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0406 18:57:11.400158 40935 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0406 18:57:11.400177 40935 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0406 18:57:11.400197 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:11.401298 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.401318 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.406490 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.406535 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.406557 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.407498 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.407529 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.407802 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.407839 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.408659 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.408685 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.413891 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.413919 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0406 18:57:11.414175 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.414200 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.414215 40935 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0406 18:57:11.414917 40935 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0406 18:57:11.414971 40935 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0406 18:57:11.414984 40935 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0406 18:57:11.415029 40935 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0406 18:57:11.415062 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:11.416202 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.416222 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.421396 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.421434 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.421440 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.422277 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.422300 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.422531 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.422560 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.423439 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.423461 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.428664 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.428689 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0406 18:57:11.428903 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.428932 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.428946 40935 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I0406 18:57:11.430133 40935 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0406 18:57:11.430190 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:11.431327 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.431353 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.431360 40935 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0406 18:57:11.431390 40935 instancegroups.go:179] waiting for 1ms after detaching instance I0406 18:57:11.432496 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.432525 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.432564 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:11.432577 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.432617 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.435257 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.435307 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.435530 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:11.435562 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.435717 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.435742 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.436462 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.537098 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.537161 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:11.537772 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:11.537806 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.538962 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.538991 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.539003 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0406 18:57:11.539176 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.539196 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.539205 40935 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0406 18:57:11.540395 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.540806 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.540831 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.541960 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.541986 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.542009 40935 instancegroups.go:407] Draining the node: "node-1e.local". I0406 18:57:11.542093 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:11.542754 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:11.542934 40935 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0406 18:57:11.542962 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.544110 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.544152 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.644780 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.644842 40935 instancegroups.go:407] Draining the node: "node-1g.local". I0406 18:57:11.644903 40935 instancegroups.go:407] Draining the node: "node-1f.local". I0406 18:57:11.645455 40935 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0406 18:57:11.645792 40935 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0406 18:57:11.645819 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.646972 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.647009 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.647038 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.647584 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.647615 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.648761 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.648799 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.749350 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.749411 40935 instancegroups.go:407] Draining the node: "node-1d.local". I0406 18:57:11.750011 40935 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0406 18:57:11.750045 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.751225 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.751274 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.751291 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0406 18:57:11.751509 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.751526 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.751535 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0406 18:57:11.751550 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0406 18:57:11.751559 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0406 18:57:11.751573 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0406 18:57:11.751667 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.751682 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.751691 40935 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0406 18:57:11.752366 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0406 18:57:11.752385 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0406 18:57:11.752394 40935 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0406 18:57:11.752420 40935 instancegroups.go:407] Draining the node: "node-1a.local". I0406 18:57:11.752923 40935 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0406 18:57:11.752945 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.754106 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.754138 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.759351 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.759428 40935 instancegroups.go:407] Draining the node: "node-1b.local". I0406 18:57:11.760058 40935 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0406 18:57:11.760086 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.761256 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.761289 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.766525 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.766580 40935 instancegroups.go:407] Draining the node: "node-1c.local". I0406 18:57:11.767180 40935 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0406 18:57:11.767207 40935 instancegroups.go:443] waiting for 1ms after terminating instance I0406 18:57:11.768361 40935 instancegroups.go:477] Validating the cluster. I0406 18:57:11.768390 40935 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0406 18:57:11.773626 40935 instancegroups.go:510] Cluster validated. I0406 18:57:11.773651 40935 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.977s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.004s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.018s === RUN TestParse E0406 18:57:09.812077 40940 versions.go:96] unable to parse Kubernetes version "" E0406 18:57:09.812215 40940 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.011s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.038s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.16s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.03s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.03s) PASS ok k8s.io/kops/pkg/kubeconfig 0.256s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.01s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.089s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.100s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.088s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0406 18:57:09.879106 40947 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.081s === RUN Test_RunEtcdManagerBuilder I0406 18:57:09.882547 40950 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0406 18:57:09.921694 40950 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W0406 18:57:09.924070 40950 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.04s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.127s === RUN Test_RunKubeApiserverBuilder I0406 18:57:09.889510 40939 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.108s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.01s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.094s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.032s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.099s === RUN TestServerGroupBuilder I0406 18:57:17.721903 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.738767 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.756713 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.793196 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.853288 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.873840 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.892067 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.911707 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.922078 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.981311 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.989389 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.000194 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.008228 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.016807 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:18.026260 41789 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.32s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.402s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.029s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.012s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.026s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.09s) --- PASS: TestIssueCert/ca (0.22s) --- PASS: TestIssueCert/client (0.33s) --- PASS: TestIssueCert/clientOneYear (0.54s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.107s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.005s === RUN TestAddUntaggedRouteTables I0406 18:57:17.721673 41797 routetable.go:53] DescribeRouteTables: { } I0406 18:57:17.721843 41797 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") W0406 18:57:17.721890 41797 tags.go:67] cluster tag not found on route-table:rtb-1234 --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0406 18:57:17.722064 41797 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0406 18:57:17.722163 41797 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0406 18:57:17.722311 41797 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0406 18:57:17.722499 41797 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0406 18:57:17.722610 41797 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0406 18:57:17.722737 41797 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0406 18:57:17.722892 41797 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0406 18:57:17.722950 41797 volumes.go:128] DescribeVolumes: { } W0406 18:57:17.722996 41797 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0406 18:57:17.723562 41797 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:17.723994 41797 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:17.724089 41797 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:17.724252 41797 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:17.724353 41797 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:17.724451 41797 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:17.724496 41797 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:17.724645 41797 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:17.724706 41797 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:17.724754 41797 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0406 18:57:17.724811 41797 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0406 18:57:17.724860 41797 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0406 18:57:17.724962 41797 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0406 18:57:17.725027 41797 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0406 18:57:17.725106 41797 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0406 18:57:17.725205 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0406 18:57:17.725289 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0406 18:57:17.725343 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0406 18:57:17.725397 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0406 18:57:17.725465 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0406 18:57:17.725526 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0406 18:57:17.725592 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0406 18:57:17.725652 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0406 18:57:17.725736 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0406 18:57:17.725798 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0406 18:57:17.725869 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0406 18:57:17.725919 41797 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.087s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.035s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.060s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.007s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.006s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.003s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.004s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.049s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.072s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0406 18:57:25.764448 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764502 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764510 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764518 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764525 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764547 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764554 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764561 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764568 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764575 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764582 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764589 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764595 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764602 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.764614 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.764624 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783155 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783205 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783273 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783285 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783338 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783349 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783404 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783415 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783469 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783480 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783537 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783548 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783622 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783634 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783710 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783722 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783774 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783786 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783843 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783855 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783921 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783933 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.783985 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.783996 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784047 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784059 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784110 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784121 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784175 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784187 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784242 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784253 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784306 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784317 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784368 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784379 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784455 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784469 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784583 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784597 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784653 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784665 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784718 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784729 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784779 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784790 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784843 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784854 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784905 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784918 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.784971 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.784982 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785034 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785046 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785096 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785108 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785160 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785172 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785235 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785248 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785303 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785314 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785369 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785381 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785433 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785444 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785494 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785506 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785556 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785567 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785640 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785653 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785721 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785734 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785799 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785811 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785863 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785874 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.785926 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.785938 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786008 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786020 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786073 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786085 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786143 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786155 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786207 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786218 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786268 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786280 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786333 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786345 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786396 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786408 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786472 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786499 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786556 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786568 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786619 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786630 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786687 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786699 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786751 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786763 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786817 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786829 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786879 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786890 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.786942 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.786954 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787006 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787018 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787069 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787080 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787133 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787145 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787198 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787209 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787260 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787271 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787320 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787332 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787383 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787395 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787468 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787480 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787535 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787547 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787597 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787608 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787661 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787672 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787739 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787752 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787803 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787814 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787883 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787897 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.787951 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.787963 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788014 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788025 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788077 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788088 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788139 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788150 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788203 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788214 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788265 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788276 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788326 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788337 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788389 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788401 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788451 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788462 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788514 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788526 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788578 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788590 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788642 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788653 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788710 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788722 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788773 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788785 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788854 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788867 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788922 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788933 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.788984 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.788995 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789046 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789628 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789690 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789702 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789753 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789765 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789816 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789828 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789879 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789890 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.789941 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.789952 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790001 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790012 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790063 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790076 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790127 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790138 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790191 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790202 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790251 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790262 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790313 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790324 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.790373 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.790384 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.03s) === RUN TestRecoversFromBadNesting W0406 18:57:25.791039 43235 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 18:57:25.791052 43235 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 18:57:25.791060 43235 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 18:57:25.791067 43235 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0406 18:57:25.791073 43235 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 18:57:25.791079 43235 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 18:57:25.791085 43235 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0406 18:57:25.791107 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791118 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791126 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.791132 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791139 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.791146 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791154 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.791161 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791168 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.791175 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791181 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791188 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791207 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791217 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.791223 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.791230 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.794676 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.794705 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.794794 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.794819 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.794910 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.794931 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795021 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795042 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795155 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795176 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795255 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795276 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795356 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795375 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795453 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795471 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795552 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795571 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795659 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795679 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795768 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795788 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795872 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795891 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.795980 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.795998 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796100 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796119 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796206 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796226 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796306 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796329 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796418 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796438 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796522 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796543 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796631 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796650 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796735 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796753 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796832 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796849 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.796928 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.796946 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797032 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797051 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797135 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797155 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797239 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797257 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797362 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797384 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797467 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797484 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797573 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797589 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797695 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797713 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797797 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797813 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.797909 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.797928 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798010 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798026 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798106 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798122 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798211 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798228 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798308 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798324 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798410 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798425 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798501 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798516 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798595 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798614 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798701 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798717 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798796 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798814 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.798898 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.798916 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799017 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799034 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799113 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799128 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799210 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799230 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799312 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799332 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799421 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799442 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799529 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799549 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799630 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799648 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799728 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799746 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799825 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799843 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.799925 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.799944 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800031 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800050 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800167 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800190 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800276 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800294 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800374 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800394 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800476 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800494 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800575 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800596 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800680 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800698 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800782 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800802 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800890 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.800910 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.800996 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801015 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801131 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801150 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801258 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801278 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801359 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801378 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801459 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801478 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801559 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801578 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801661 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801681 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801770 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801790 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801877 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.801896 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.801973 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802009 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802090 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802108 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802189 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802207 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802285 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802303 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802385 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802403 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802486 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802503 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802586 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802605 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802739 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802758 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802839 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802856 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.802937 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.802958 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803039 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803057 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803139 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803157 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803238 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803256 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803338 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803356 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803441 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803458 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803542 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803562 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803644 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803663 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803743 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803763 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803843 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803861 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.803942 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.803963 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804049 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804068 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804147 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804166 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804255 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804274 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804355 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804374 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804459 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804478 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804572 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804592 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804674 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804695 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804778 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804797 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804879 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804897 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0406 18:57:25.804977 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0406 18:57:25.804996 43235 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.01s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.045s === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.070s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.028s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.00s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.01s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.045s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.03s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.02s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.00s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.00s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.094s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0406 18:57:28.290531 43724 files.go:57] Writing file "/tmp/TestWriteFile2394701340/001/SubDir/test1.tmp" I0406 18:57:28.290754 43724 files.go:113] Changing file mode for "/tmp/TestWriteFile2394701340/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.01s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.049s ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestBootstrapChannelBuilder_BuildTasks I0406 18:57:28.755786 43869 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:28.756156 43869 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:28.756231 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:28.756302 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:28.756381 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:28.756444 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:28.756530 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:28.756650 43869 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:28.756723 43869 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:28.774742 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:28.851583 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.025853 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.110401 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.181299 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.249242 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.335245 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.400408 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.506046 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:29.939444 43869 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.24s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0406 18:57:29.993396 43869 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:29.993572 43869 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:29.993646 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:29.993681 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:29.993715 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:29.993743 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:29.993759 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:29.993781 43869 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:29.993797 43869 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:29.993817 43869 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0406 18:57:29.995065 43869 zones.go:93] ListHostedZonesPages { } I0406 18:57:30.051490 43869 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.06s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0406 18:57:30.051715 43869 featureflag.go:160] FeatureFlag "ImageDigest"=false I0406 18:57:30.051880 43869 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:30.051940 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0406 18:57:30.051975 43869 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0406 18:57:30.052006 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0406 18:57:30.052034 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0406 18:57:30.052051 43869 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0406 18:57:30.052073 43869 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0406 18:57:30.052091 43869 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0406 18:57:30.053324 43869 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.09s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.07s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0406 18:57:30.244922 43869 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.00s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0406 18:57:30.280303 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280340 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280357 43869 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0406 18:57:30.280391 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0406 18:57:30.280407 43869 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0406 18:57:30.280439 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0406 18:57:30.280469 43869 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0406 18:57:30.280497 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280516 43869 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0406 18:57:30.280545 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280563 43869 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0406 18:57:30.280591 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280629 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280659 43869 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0406 18:57:30.280676 43869 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0406 18:57:30.280691 43869 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0406 18:57:30.280705 43869 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0406 18:57:30.280721 43869 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0406 18:57:30.280736 43869 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0406 18:57:30.280753 43869 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0406 18:57:30.280769 43869 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0406 18:57:30.282564 43869 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.641s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0406 18:57:28.733382 43871 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3862849654/001 I0406 18:57:28.735954 43871 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender3862849654/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0406 18:57:28.742861 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0406 18:57:28.742958 43871 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0406 18:57:28.743030 43871 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0406 18:57:28.743275 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.743456 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.743518 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.743535 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.743701 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.743736 43871 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 18:57:28.743769 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.743836 43871 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 18:57:28.743917 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 18:57:28.744067 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.744111 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.744164 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.744181 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.744221 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.744280 43871 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 18:57:28.744339 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0406 18:57:28.744678 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.744736 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.744840 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.744876 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.744907 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.744932 43871 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0406 18:57:28.745054 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.745278 43871 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 18:57:28.745328 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.745351 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.745374 43871 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0406 18:57:28.745435 43871 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.745582 43871 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0406 18:57:28.745640 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.745665 43871 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0406 18:57:28.745700 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 18:57:28.745851 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.745894 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.745936 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.745950 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.745978 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.746000 43871 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0406 18:57:28.746048 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.746100 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.746124 43871 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0406 18:57:28.746157 43871 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.746206 43871 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0406 18:57:28.746239 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0406 18:57:28.746276 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0406 18:57:28.746428 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0406 18:57:28.746472 43871 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0406 18:57:28.746493 43871 internetgateways.go:66] CreateInternetGateway: { } I0406 18:57:28.746509 43871 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0406 18:57:28.746630 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.746669 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.746706 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.746721 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.746790 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.746814 43871 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 18:57:28.746843 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.746897 43871 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 18:57:28.746964 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 18:57:28.747084 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.747122 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.747158 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.747172 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.747205 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.747238 43871 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0406 18:57:28.747287 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0406 18:57:28.749129 43871 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1875066029/001 I0406 18:57:28.751106 43871 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1875066029/001 --- PASS: TestLaunchTemplateTerraformRender (0.00s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0406 18:57:28.752139 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.752232 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.752397 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.752468 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.752512 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.752556 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.752659 43871 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0406 18:57:28.752957 43871 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0406 18:57:28.753029 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0406 18:57:28.753080 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 18:57:28.753279 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.753351 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.753432 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.753463 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.753510 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.753550 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.753618 43871 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0406 18:57:28.753739 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0406 18:57:28.754355 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.754427 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.754597 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.754664 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.754703 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.754745 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.754812 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.754975 43871 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 18:57:28.755042 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.755087 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.755132 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 18:57:28.755338 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.755411 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.755485 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.755513 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.755555 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.755600 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.755659 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.755756 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.755817 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0406 18:57:28.756375 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.756522 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.756689 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.756762 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.756802 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.756857 43871 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 18:57:28.756933 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.757027 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.757095 43871 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 18:57:28.757171 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.757209 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.757273 43871 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:28.757339 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 18:57:28.757581 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.757650 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.757727 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.757757 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.757800 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.757841 43871 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 18:57:28.757918 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.758015 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.758054 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.758136 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0406 18:57:28.758538 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.758604 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.758738 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.758803 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.758846 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.758886 43871 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 18:57:28.758951 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.759106 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.759213 43871 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0406 18:57:28.759253 43871 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0406 18:57:28.759317 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.759400 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.759492 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0406 18:57:28.759737 43871 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0406 18:57:28.759808 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.759889 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.759928 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.759974 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.760015 43871 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0406 18:57:28.760074 43871 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0406 18:57:28.760216 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.760287 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.760361 43871 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0406 18:57:28.760581 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0406 18:57:28.760680 43871 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0406 18:57:28.760892 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.760950 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.761012 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.761042 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.761165 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.761227 43871 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 18:57:28.761275 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.761326 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.761490 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0406 18:57:28.761529 43871 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0406 18:57:28.761580 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0406 18:57:28.761784 43871 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0406 18:57:28.761849 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.761911 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.761936 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.761980 43871 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0406 18:57:28.762034 43871 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0406 18:57:28.762118 43871 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0406 18:57:28.762474 43871 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:28.762532 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.762690 43871 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0406 18:57:28.762752 43871 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0406 18:57:28.762786 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.762816 43871 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:28.762980 43871 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:28.763089 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0406 18:57:28.763159 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.763188 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.763230 43871 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0406 18:57:28.763284 43871 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0406 18:57:28.763652 43871 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0406 18:57:28.763846 43871 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:28.763946 43871 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0406 18:57:28.764065 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0406 18:57:28.764127 43871 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0406 18:57:28.764223 43871 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.095s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.068s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.029s === RUN TestDiskRenderAzure I0406 18:57:28.708514 43874 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0406 18:57:28.709173 43874 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0406 18:57:28.709631 43874 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0406 18:57:28.710368 43874 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0406 18:57:28.710797 43874 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0406 18:57:28.711295 43874 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0406 18:57:28.711754 43874 resourcegroup.go:103] Creating a new Resource Group with name: rg I0406 18:57:28.711777 43874 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0406 18:57:28.712088 43874 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0406 18:57:28.713092 43874 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0406 18:57:28.713564 43874 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0406 18:57:28.713979 43874 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0406 18:57:28.714564 43874 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0406 18:57:28.715005 43874 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.054s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.032s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestProjectIAMBinding I0406 18:57:31.541018 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.541533 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.541664 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.542049 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.542175 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.542301 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0406 18:57:31.542483 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.542569 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.542678 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.542781 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.542885 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.542926 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0406 18:57:31.543098 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.543325 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.543426 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.543701 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0406 18:57:31.543804 44920 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0406 18:57:31.543903 44920 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.050s === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.034s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.031s === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.032s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.006s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.024s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0406 18:57:34.819822 45176 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0406 18:57:34.819905 45176 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0406 18:57:34.819918 45176 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0406 18:57:34.819944 45176 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0406 18:57:34.819956 45176 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0406 18:57:34.819969 45176 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0406 18:57:34.819982 45176 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0406 18:57:34.820625 45176 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.057s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.005s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.006s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.004s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.006s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.007s === RUN TestGetProxyEnvVars W0406 18:57:34.789433 45180 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.023s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.007s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.005s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.005s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.025s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0406 18:57:37.584414 45269 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.053s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 134.4 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r4.apk >>> kops: Build complete at Thu, 06 Apr 2023 18:57:41 +0000 elapsed time 0h 3m 22s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20230406.185420) (2/3) Purging kubernetes (1.26.1-r3) (3/3) Purging go (1.20.3-r0) Executing busybox-1.36.0-r5.trigger OK: 395 MiB in 101 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...