>>> kops: Building testing/kops 1.25.3-r3 (using abuild 3.10.0-r0) started Thu, 16 Feb 2023 02:03:15 +0000 >>> kops: Checking sanity of /home/buildozer/aports/testing/kops/APKBUILD... >>> kops: Analyzing dependencies... >>> kops: Installing for build: build-base kubernetes go (1/3) Installing kubernetes (1.26.1-r2) (2/3) Installing go (1.20.1-r0) (3/3) Installing .makedepends-kops (20230216.020317) Executing busybox-1.36.0-r4.trigger OK: 562 MiB in 101 packages >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 54 31.1M 54 16.9M 0 0 43.1M 0 --:--:-- --:--:-- --:--:-- 43.1M 100 31.1M 100 31.1M 0 0 57.3M 0 --:--:-- --:--:-- --:--:-- 57.2M >>> kops: Fetching https://distfiles.alpinelinux.org/distfiles/edge/kops-1.25.3.tar.gz >>> kops: Checking sha512sums... kops-1.25.3.tar.gz: OK >>> kops: Unpacking /var/cache/distfiles/edge/kops-1.25.3.tar.gz... go: downloading github.com/cert-manager/cert-manager v1.10.0 go: downloading github.com/blang/semver/v4 v4.0.0 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/spf13/viper v1.14.0 go: downloading go.uber.org/multierr v1.8.0 go: downloading k8s.io/klog/v2 v2.80.1 go: downloading github.com/aws/aws-sdk-go v1.44.139 go: downloading k8s.io/apimachinery v0.25.4 go: downloading google.golang.org/api v0.103.0 go: downloading k8s.io/api v0.25.4 go: downloading github.com/gophercloud/gophercloud v1.0.0 go: downloading k8s.io/client-go v0.25.4 go: downloading github.com/google/uuid v1.3.0 go: downloading k8s.io/klog v1.0.0 go: downloading github.com/aws/amazon-ec2-instance-selector/v2 v2.4.0 go: downloading github.com/go-logr/logr v1.2.3 go: downloading k8s.io/cli-runtime v0.25.4 go: downloading github.com/prometheus/client_golang v1.14.0 go: downloading github.com/google/go-containerregistry v0.12.1 go: downloading github.com/spf13/pflag v1.0.5 go: downloading github.com/digitalocean/godo v1.90.0 go: downloading cloud.google.com/go/compute/metadata v0.2.1 go: downloading cloud.google.com/go/compute v1.12.1 go: downloading k8s.io/component-base v0.25.4 go: downloading golang.org/x/oauth2 v0.2.0 go: downloading github.com/google/go-cmp v0.5.9 go: downloading golang.org/x/crypto v0.3.0 go: downloading gopkg.in/gcfg.v1 v1.2.3 go: downloading sigs.k8s.io/controller-runtime v0.13.1 go: downloading helm.sh/helm/v3 v3.10.2 go: downloading k8s.io/legacy-cloud-providers v0.25.4 go: downloading sigs.k8s.io/yaml v1.3.0 go: downloading k8s.io/kubectl v0.25.4 go: downloading go.uber.org/atomic v1.9.0 go: downloading github.com/stretchr/testify v1.8.1 go: downloading github.com/inconshreveable/mousetrap v1.0.1 go: downloading github.com/hetznercloud/hcloud-go v1.37.0 go: downloading github.com/pelletier/go-toml v1.9.5 go: downloading k8s.io/kubelet v0.25.4 go: downloading k8s.io/mount-utils v0.25.4 go: downloading k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 go: downloading github.com/fsnotify/fsnotify v1.6.0 go: downloading github.com/mitchellh/mapstructure v1.5.0 go: downloading github.com/spf13/afero v1.9.2 go: downloading github.com/spf13/cast v1.5.0 go: downloading github.com/spf13/jwalterweatherman v1.1.0 go: downloading github.com/gogo/protobuf v1.3.2 go: downloading github.com/google/gofuzz v1.2.0 go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.2.3 go: downloading golang.org/x/sync v0.1.0 go: downloading k8s.io/cloud-provider-aws v1.25.1 go: downloading golang.org/x/sys v0.2.0 go: downloading gopkg.in/inf.v0 v0.9.1 go: downloading github.com/davecgh/go-spew v1.1.1 go: downloading github.com/google/gnostic v0.6.9 go: downloading github.com/prometheus/client_model v0.3.0 go: downloading github.com/prometheus/common v0.37.0 go: downloading google.golang.org/protobuf v1.28.1 go: downloading github.com/mitchellh/go-homedir v1.1.0 go: downloading github.com/imdario/mergo v0.3.12 go: downloading golang.org/x/net v0.2.0 go: downloading github.com/evanphx/json-patch v5.6.0+incompatible go: downloading github.com/Azure/azure-sdk-for-go v67.1.0+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.11 go: downloading github.com/Azure/go-autorest/autorest v0.11.28 go: downloading github.com/google/go-querystring v1.1.0 go: downloading github.com/Azure/go-autorest v14.2.0+incompatible go: downloading golang.org/x/time v0.0.0-20220922220347-f3bd1da661af go: downloading golang.org/x/term v0.2.0 go: downloading github.com/apparentlymart/go-cidr v1.1.0 go: downloading github.com/MakeNowJust/heredoc/v2 v2.0.1 go: downloading github.com/cpuguy83/go-md2man/v2 v2.0.2 go: downloading github.com/MakeNowJust/heredoc v1.0.0 go: downloading gopkg.in/yaml.v3 v3.0.1 go: downloading google.golang.org/appengine v1.6.7 go: downloading github.com/Azure/azure-pipeline-go v0.2.3 go: downloading github.com/Azure/azure-storage-blob-go v0.15.0 go: downloading github.com/go-ini/ini v1.67.0 go: downloading github.com/hashicorp/vault/api v1.8.2 go: downloading github.com/pkg/sftp v1.13.5 go: downloading github.com/opencontainers/go-digest v1.0.0 go: downloading github.com/Masterminds/sprig/v3 v3.2.2 go: downloading gopkg.in/warnings.v0 v0.1.2 go: downloading github.com/evanphx/json-patch/v5 v5.6.0 go: downloading github.com/pkg/errors v0.9.1 go: downloading github.com/jacksontj/memberlistmesh v0.0.0-20190905163944-93462b9d2bb7 go: downloading github.com/weaveworks/mesh v0.0.0-20191105120815-58dbcc3e8e63 go: downloading gopkg.in/yaml.v2 v2.4.0 go: downloading github.com/sergi/go-diff v1.2.0 go: downloading github.com/GoogleCloudPlatform/k8s-cloud-provider v1.18.1-0.20220218231025-f11817397a1b go: downloading k8s.io/cloud-provider v0.25.4 go: downloading github.com/chai2010/gettext-go v1.0.2 go: downloading github.com/mitchellh/go-wordwrap v1.0.0 go: downloading github.com/russross/blackfriday v1.6.0 go: downloading gopkg.in/square/go-jose.v2 v2.6.0 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0 go: downloading github.com/spotinst/spotinst-sdk-go v1.131.1 go: downloading github.com/google/go-tpm-tools v0.3.9 go: downloading github.com/google/go-tpm v0.3.3 go: downloading github.com/hashicorp/hcl/v2 v2.15.0 go: downloading github.com/zclconf/go-cty v1.12.1 go: downloading k8s.io/gengo v0.0.0-20221011193443-fad74ee6edd9 go: downloading github.com/pmezard/go-difflib v1.0.0 go: downloading github.com/moby/sys/mountinfo v0.6.0 go: downloading github.com/subosito/gotenv v1.4.1 go: downloading github.com/hashicorp/hcl v1.0.0 go: downloading gopkg.in/ini.v1 v1.67.0 go: downloading github.com/magiconair/properties v1.8.6 go: downloading github.com/pelletier/go-toml/v2 v2.0.5 go: downloading golang.org/x/text v0.4.0 go: downloading sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 go: downloading github.com/json-iterator/go v1.1.12 go: downloading github.com/golang/protobuf v1.5.2 go: downloading k8s.io/apiextensions-apiserver v0.25.2 go: downloading sigs.k8s.io/gateway-api v0.5.0 go: downloading google.golang.org/grpc v1.50.1 go: downloading github.com/googleapis/gax-go/v2 v2.7.0 go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.21 go: downloading k8s.io/kube-openapi v0.0.0-20220803164354-a70c9af30aea go: downloading k8s.io/csi-translation-lib v0.25.4 go: downloading github.com/beorn7/perks v1.0.1 go: downloading github.com/cespare/xxhash/v2 v2.1.2 go: downloading github.com/prometheus/procfs v0.8.0 go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 go: downloading github.com/patrickmn/go-cache v2.1.0+incompatible go: downloading github.com/charmbracelet/bubbles v0.13.0 go: downloading github.com/charmbracelet/bubbletea v0.21.0 go: downloading github.com/charmbracelet/lipgloss v0.5.0 go: downloading github.com/evertras/bubble-table v0.14.4 go: downloading github.com/muesli/termenv v0.11.1-0.20220212125758-44cd13922739 go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de go: downloading sigs.k8s.io/kustomize/api v0.12.1 go: downloading sigs.k8s.io/kustomize/kyaml v0.13.9 go: downloading github.com/gregjones/httpcache v0.0.0-20190611155906-901d90724c79 go: downloading github.com/peterbourgon/diskv v2.0.1+incompatible go: downloading github.com/Azure/go-autorest/autorest/azure/cli v0.4.5 go: downloading github.com/Azure/go-autorest/logger v0.2.1 go: downloading github.com/dimchansky/utfbom v1.1.1 go: downloading github.com/russross/blackfriday/v2 v2.1.0 go: downloading github.com/mattn/go-ieproxy v0.0.1 go: downloading github.com/docker/cli v20.10.20+incompatible go: downloading github.com/docker/distribution v2.8.1+incompatible go: downloading github.com/containerd/stargz-snapshotter/estargz v0.12.1 go: downloading github.com/onsi/ginkgo v1.16.5 go: downloading github.com/onsi/gomega v1.20.2 go: downloading go.uber.org/goleak v1.2.0 go: downloading github.com/frankban/quicktest v1.14.3 go: downloading gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c go: downloading github.com/go-logr/zapr v1.2.3 go: downloading go.uber.org/zap v1.21.0 go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da go: downloading gomodules.xyz/jsonpatch/v2 v2.2.0 go: downloading github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 go: downloading github.com/Masterminds/goutils v1.1.1 go: downloading github.com/Masterminds/semver/v3 v3.1.1 go: downloading github.com/huandu/xstrings v1.3.2 go: downloading github.com/mitchellh/copystructure v1.2.0 go: downloading github.com/shopspring/decimal v1.2.0 go: downloading github.com/cenkalti/backoff/v3 v3.0.0 go: downloading github.com/hashicorp/errwrap v1.1.0 go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 go: downloading github.com/hashicorp/go-multierror v1.1.1 go: downloading github.com/hashicorp/go-retryablehttp v0.7.1 go: downloading github.com/hashicorp/go-rootcerts v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 go: downloading github.com/hashicorp/vault/sdk v0.6.0 go: downloading github.com/go-test/deep v1.0.3 go: downloading github.com/hashicorp/go-hclog v1.2.0 go: downloading github.com/kr/fs v0.1.0 go: downloading github.com/Azure/go-autorest/tracing v0.6.0 go: downloading github.com/hashicorp/go-sockaddr v1.0.2 go: downloading github.com/Azure/go-autorest/autorest/mocks v0.4.2 go: downloading github.com/hashicorp/memberlist v0.3.1 go: downloading github.com/oklog/ulid v1.3.1 go: downloading golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 go: downloading google.golang.org/genproto v0.0.0-20221027153422-115e99e71e1c go: downloading go.opencensus.io v0.24.0 go: downloading golang.org/x/tools v0.1.8 go: downloading github.com/stretchr/objx v0.5.0 go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0 go: downloading github.com/golang-jwt/jwt/v4 v4.2.0 go: downloading github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 go: downloading github.com/agext/levenshtein v1.2.1 go: downloading github.com/apparentlymart/go-textseg/v13 v13.0.0 go: downloading github.com/kylelemons/godebug v1.1.0 go: downloading github.com/kr/pretty v0.3.0 go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd go: downloading github.com/modern-go/reflect2 v1.0.2 go: downloading github.com/lucasb-eyer/go-colorful v1.2.0 go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading github.com/muesli/reflow v0.3.0 go: downloading github.com/sahilm/fuzzy v0.1.0 go: downloading github.com/atotto/clipboard v0.1.4 go: downloading github.com/containerd/console v1.0.3 go: downloading github.com/mattn/go-isatty v0.0.14 go: downloading github.com/muesli/ansi v0.0.0-20211018074035-2e021307bc4b go: downloading github.com/muesli/cancelreader v0.2.0 go: downloading github.com/google/btree v1.0.1 go: downloading github.com/jmespath/go-jmespath v0.4.0 go: downloading github.com/opencontainers/image-spec v1.1.0-rc2 go: downloading github.com/klauspost/compress v1.15.11 go: downloading github.com/vbatts/tar-split v0.11.2 go: downloading github.com/google/go-attestation v0.4.4-0.20220404204839-8820d49b18d9 go: downloading github.com/containerd/containerd v1.6.6 go: downloading github.com/sirupsen/logrus v1.9.0 go: downloading oras.land/oras-go v1.2.0 go: downloading github.com/distribution/distribution/v3 v3.0.0-20220526142353-ffbd94cbe269 go: downloading github.com/phayes/freeport v0.0.0-20220201140144-74d24b5ae9f5 go: downloading github.com/onsi/ginkgo/v2 v2.2.0 go: downloading github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d go: downloading github.com/creack/pty v1.1.11 go: downloading gotest.tools/v3 v3.0.3 go: downloading github.com/mitchellh/reflectwalk v1.0.2 go: downloading github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 go: downloading github.com/fatih/color v1.13.0 go: downloading github.com/mattn/go-colorable v0.1.12 go: downloading github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 go: downloading github.com/go-openapi/swag v0.19.14 go: downloading github.com/googleapis/enterprise-certificate-proxy v0.2.0 go: downloading github.com/armon/go-metrics v0.4.0 go: downloading github.com/hashicorp/go-msgpack v0.5.3 go: downloading github.com/miekg/dns v1.1.50 go: downloading github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 go: downloading github.com/kr/text v0.2.0 go: downloading github.com/rogpeppe/go-internal v1.8.1 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading github.com/go-errors/errors v1.0.1 go: downloading github.com/mitchellh/go-testing-interface v1.0.0 go: downloading github.com/docker/docker v20.10.20+incompatible go: downloading github.com/docker/go-connections v0.4.0 go: downloading github.com/go-openapi/jsonreference v0.19.5 go: downloading github.com/moby/spdystream v0.2.0 go: downloading github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 go: downloading github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 go: downloading github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 go: downloading github.com/nxadm/tail v1.4.8 go: downloading github.com/ryanuber/go-glob v1.0.0 go: downloading github.com/emicklei/go-restful/v3 v3.8.0 go: downloading github.com/mailru/easyjson v0.7.6 go: downloading github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad go: downloading github.com/jmespath/go-jmespath/internal/testify v1.5.1 go: downloading github.com/hashicorp/go-immutable-radix v1.3.1 go: downloading github.com/pascaldekloe/goe v0.1.0 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 go: downloading github.com/xlab/treeprint v1.1.0 go: downloading github.com/golang/snappy v0.0.4 go: downloading github.com/pierrec/lz4 v2.5.2+incompatible go: downloading github.com/hashicorp/go-plugin v1.4.5 go: downloading github.com/hashicorp/go-secure-stdlib/mlock v0.1.1 go: downloading github.com/hashicorp/go-uuid v1.0.2 go: downloading github.com/hashicorp/go-version v1.2.1 go: downloading github.com/hashicorp/golang-lru v0.5.4 go: downloading github.com/armon/go-radix v1.0.0 go: downloading github.com/docker/docker-credential-helpers v0.7.0 go: downloading github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d go: downloading github.com/bshuster-repo/logrus-logstash-hook v1.0.0 go: downloading github.com/bugsnag/bugsnag-go v0.0.0-20141110184014-b1d153021fcd go: downloading github.com/docker/go-metrics v0.0.1 go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1 go: downloading github.com/gorilla/handlers v1.5.1 go: downloading github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50 go: downloading github.com/PuerkitoBio/purell v1.1.1 go: downloading github.com/go-openapi/jsonpointer v0.19.5 go: downloading gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 go: downloading github.com/josharian/intern v1.0.0 go: downloading github.com/google/certificate-transparency-go v1.1.2 go: downloading github.com/fatih/structs v1.1.0 go: downloading github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb go: downloading github.com/oklog/run v1.0.0 go: downloading github.com/jhump/protoreflect v1.6.0 go: downloading github.com/gorilla/mux v1.8.0 go: downloading github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c go: downloading github.com/gomodule/redigo v1.8.2 go: downloading github.com/bugsnag/panicwrap v0.0.0-20151223152923-e2c28503fcd0 go: downloading github.com/felixge/httpsnoop v1.0.1 go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43 go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 go: downloading github.com/moby/locker v1.0.1 go: downloading go.starlark.net v0.0.0-20200306205701-8dd3e2ee1dd5 go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b go: downloading github.com/Microsoft/hcsshim v0.9.3 go: downloading golang.org/x/mod v0.6.0 go: downloading github.com/Microsoft/go-winio v0.6.0 go: downloading github.com/containerd/cgroups v1.0.3 go: downloading github.com/docker/go-units v0.5.0 go: downloading github.com/morikuni/aec v1.0.0 go: downloading cloud.google.com/go/longrunning v0.1.1 go: downloading cloud.google.com/go v0.105.0 go: downloading github.com/Azure/go-autorest/autorest/validation v0.3.1 fatal: not a git repository (or any of the parent directories): .git /bin/sh: tools/get_version.sh: not found mkdir -p /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64 GOOS=linux GOARCH=amd64 go build "-trimpath" -o /home/buildozer/aports/testing/kops/src/kops-1.25.3/.build/dist/linux/amd64/kops -ldflags=all=" -X k8s.io/kops.Version= -X k8s.io/kops.GitVersion=" k8s.io/kops/cmd/kops fatal: not a git repository (or any of the parent directories): .git go test -v ./... ? k8s.io/kops [no test files] ? k8s.io/kops/channels/cmd/channels [no test files] ? k8s.io/kops/channels/pkg/api [no test files] ? k8s.io/kops/cloudmock/aws/mockautoscaling [no test files] ? k8s.io/kops/cloudmock/aws/mockec2 [no test files] ? k8s.io/kops/cloudmock/aws/mockelb [no test files] ? k8s.io/kops/cloudmock/aws/mockelbv2 [no test files] ? k8s.io/kops/cloudmock/aws/mockeventbridge [no test files] ? k8s.io/kops/cloudmock/aws/mockiam [no test files] ? k8s.io/kops/cloudmock/aws/mockroute53 [no test files] ? k8s.io/kops/cloudmock/aws/mocksqs [no test files] ? k8s.io/kops/cloudmock/gce [no test files] ? k8s.io/kops/cloudmock/gce/gcphttp [no test files] ? k8s.io/kops/cloudmock/gce/mockcloudresourcemanager [no test files] ? k8s.io/kops/cloudmock/gce/mockcompute [no test files] ? k8s.io/kops/cloudmock/gce/mockdns [no test files] ? k8s.io/kops/cloudmock/gce/mockiam [no test files] ? k8s.io/kops/cloudmock/gce/mockstorage [no test files] ? k8s.io/kops/cloudmock/openstack [no test files] ? k8s.io/kops/cloudmock/openstack/mockblockstorage [no test files] ? k8s.io/kops/cloudmock/openstack/mockcompute [no test files] ? k8s.io/kops/cloudmock/openstack/mockimage [no test files] ? k8s.io/kops/cloudmock/openstack/mockdns [no test files] ? k8s.io/kops/cloudmock/openstack/mockloadbalancer [no test files] ? k8s.io/kops/cloudmock/openstack/mocknetworking [no test files] === RUN Test_Filtering --- PASS: Test_Filtering (0.00s) === RUN Test_Replacement --- PASS: Test_Replacement (0.00s) === RUN Test_GetRequiredUpdates --- PASS: Test_GetRequiredUpdates (0.00s) === RUN Test_NeedsRollingUpdate I0216 02:05:34.925909 45032 addon.go:233] addon test wants to update all nodes I0216 02:05:34.928384 45032 addon.go:233] addon test wants to update worker nodes I0216 02:05:34.928890 45032 addon.go:233] addon test wants to update control-plane nodes --- PASS: Test_NeedsRollingUpdate (0.00s) === RUN Test_InstallPKI I0216 02:05:34.929506 45032 addon.go:266] installing PKI for "test" I0216 02:05:35.755456 45032 addon.go:266] installing PKI for "test" --- PASS: Test_InstallPKI (1.23s) === RUN Test_IsPKIInstalled --- PASS: Test_IsPKIInstalled (0.00s) PASS ok k8s.io/kops/channels/pkg/channels 1.304s === RUN TestGetUpdates --- PASS: TestGetUpdates (0.00s) PASS ok k8s.io/kops/channels/pkg/cmd 0.078s ? k8s.io/kops/cmd/kops/util [no test files] ? k8s.io/kops/cmd/kops-controller [no test files] ? k8s.io/kops/cmd/kops-controller/controllers [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/server [no test files] ? k8s.io/kops/cmd/kops-controller/pkg/config [no test files] ? k8s.io/kops/cmd/nodeup [no test files] ? k8s.io/kops/dns-controller/cmd/dns-controller [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/interfaces [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns/internal/stubs [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/rrstype [no test files] ? k8s.io/kops/dnsprovider/pkg/dnsprovider/tests [no test files] ? k8s.io/kops/examples/kops-api-example [no test files] ? k8s.io/kops/nodeup/pkg/model/dns [no test files] ? k8s.io/kops/nodeup/pkg/model/networking [no test files] ? k8s.io/kops/nodeup/pkg/model/resources [no test files] ? k8s.io/kops/pkg/acls [no test files] ? k8s.io/kops/pkg/acls/gce [no test files] ? k8s.io/kops/pkg/apis/kops/registry [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/apis/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/apis/nodeup [no test files] ? k8s.io/kops/pkg/applylib [no test files] ? k8s.io/kops/pkg/applylib/mocks [no test files] ? k8s.io/kops/pkg/applylib/mocks/mockkubeapiserver [no test files] ? k8s.io/kops/pkg/bootstrap [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/clientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/scheme [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/internalversion/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha2/fake [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3 [no test files] ? k8s.io/kops/pkg/client/clientset_generated/internalclientset/typed/kops/v1alpha3/fake [no test files] ? k8s.io/kops/pkg/client/simple [no test files] ? k8s.io/kops/pkg/client/simple/api [no test files] ? k8s.io/kops/pkg/clouds [no test files] ? k8s.io/kops/pkg/clusteraddons [no test files] ? k8s.io/kops/pkg/commands/commandutils [no test files] ? k8s.io/kops/pkg/commands/helpers [no test files] ? k8s.io/kops/pkg/configserver [no test files] ? k8s.io/kops/pkg/dump [no test files] ? k8s.io/kops/pkg/kubemanifest [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsloadbalancercontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/karpenter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awsebscsidriver [no test files] ? k8s.io/kops/pkg/model/components/kopscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/dnscontroller [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/externaldns [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/kuberouter [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/certmanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/nodeterminationhandler [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/awscloudcontrollermanager [no test files] ? k8s.io/kops/pkg/model/components/addonmanifests/clusterautoscaler [no test files] ? k8s.io/kops/pkg/model/domodel [no test files] ? k8s.io/kops/pkg/model/gcemodel [no test files] ? k8s.io/kops/pkg/model/hetznermodel [no test files] ? k8s.io/kops/pkg/nodeidentity [no test files] ? k8s.io/kops/pkg/nodeidentity/aws [no test files] ? k8s.io/kops/pkg/nodeidentity/do [no test files] ? k8s.io/kops/pkg/nodeidentity/gce [no test files] ? k8s.io/kops/pkg/nodeidentity/hetzner [no test files] ? k8s.io/kops/pkg/nodeidentity/openstack [no test files] ? k8s.io/kops/pkg/rbac [no test files] ? k8s.io/kops/pkg/resolver [no test files] ? k8s.io/kops/pkg/resources [no test files] ? k8s.io/kops/pkg/resources/digitalocean [no test files] ? k8s.io/kops/pkg/resources/hetzner [no test files] ? k8s.io/kops/pkg/resources/openstack [no test files] ? k8s.io/kops/pkg/resources/ops [no test files] ? k8s.io/kops/pkg/resources/spotinst [no test files] ? k8s.io/kops/pkg/sshcredentials [no test files] ? k8s.io/kops/pkg/templates [no test files] ? k8s.io/kops/pkg/testutils [no test files] ? k8s.io/kops/pkg/testutils/golden [no test files] ? k8s.io/kops/pkg/tokens [no test files] ? k8s.io/kops/pkg/try [no test files] ? k8s.io/kops/pkg/values [no test files] ? k8s.io/kops/pkg/wellknownoperators [no test files] ? k8s.io/kops/pkg/wellknownports [no test files] ? k8s.io/kops/pkg/wellknownusers [no test files] ? k8s.io/kops/pkg/zones [no test files] ? k8s.io/kops/protokube/pkg/gossip [no test files] ? k8s.io/kops/protokube/pkg/etcd [no test files] ? k8s.io/kops/protokube/pkg/gossip/aws [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns [no test files] ? k8s.io/kops/protokube/pkg/gossip/dns/provider [no test files] ? k8s.io/kops/protokube/cmd/protokube [no test files] ? k8s.io/kops/protokube/pkg/gossip/mesh [no test files] ? k8s.io/kops/protokube/pkg/gossip/do [no test files] ? k8s.io/kops/protokube/pkg/gossip/memberlist [no test files] ? k8s.io/kops/protokube/pkg/gossip/openstack [no test files] ? k8s.io/kops/protokube/pkg/gossip/hetzner [no test files] ? k8s.io/kops/tests/fuzz [no test files] ? k8s.io/kops/tests/integration/channel/simple [no test files] ? k8s.io/kops/upup/models [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/bootstrapchannelbuilder [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/do [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcediscovery [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/gcemetadata [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/cloudformation [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmsigner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/gce/tpm/gcetpmverifier [no test files] === RUN TestCreateClusterMinimal I0216 02:05:36.995242 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:36.995498 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:36.995544 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:36.995585 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:36.995649 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:36.995698 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:36.995714 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:36.995748 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:36.995778 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:36.995859 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:36.995893 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.047535 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.047845 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.048509 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.065137 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.065317 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.065354 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.065418 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.065453 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.065481 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.065501 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.065529 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.065547 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.065609 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.065634 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.153256 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.153425 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.153724 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.160881 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.161041 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.161076 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.161107 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.161139 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.161165 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.161182 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.161206 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.161223 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.161279 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.161304 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.193245 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.193421 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.193710 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.199241 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.199384 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.199416 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.199458 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.199506 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.199536 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.199557 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.199580 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.199597 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.199655 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.199678 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.233156 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.233334 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.233619 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.239198 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.239336 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.239366 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.239395 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.239425 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.239451 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.239467 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.239489 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.239507 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.239584 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.239612 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.334374 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.334543 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.335616 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.346321 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.346566 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.346620 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.346676 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.346741 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.346788 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.346823 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.346866 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.346897 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.346996 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.347042 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.377721 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.378042 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.378529 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.387761 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.387959 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.388004 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.388042 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.388089 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.388128 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.388176 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.388217 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.388245 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.388374 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.388420 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.501471 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.501648 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.501946 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.507603 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.507772 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.507809 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.507838 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.507870 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.507897 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.507913 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.507936 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.507954 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.508016 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.508040 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.539733 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.539880 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.540327 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterMinimal (0.55s) === RUN TestCreateClusterHetzner I0216 02:05:37.550152 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.550434 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.550485 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.550538 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.550589 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.550649 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.550693 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.550718 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.550753 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.550873 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.550919 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0216 02:05:37.610754 46260 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0216 02:05:37.610791 46260 new_cluster.go:1279] Cloud Provider ID = hetzner I0216 02:05:37.617081 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.617201 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.617227 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.617251 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.617276 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.617315 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.617337 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.617357 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.617372 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.617416 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.617439 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.685735 46260 new_cluster.go:1279] Cloud Provider ID = hetzner --- PASS: TestCreateClusterHetzner (0.14s) === RUN TestCreateClusterOverride I0216 02:05:37.691784 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.692041 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.692091 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.692167 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.692548 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.692664 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.692865 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.692908 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.692926 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.692979 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.693003 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.764734 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.765207 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.765467 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterOverride (0.08s) === RUN TestCreateClusterComplex I0216 02:05:37.770644 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.770789 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.770817 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.770843 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.770869 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.770893 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.770911 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.770934 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.770950 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.771010 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.771034 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.808303 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.808440 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.808641 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterComplex (0.04s) === RUN TestCreateClusterHA I0216 02:05:37.813576 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.813709 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.813735 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.813759 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.813785 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.813807 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.813824 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.813844 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.813859 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.813910 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.813933 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.845452 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.845749 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.845763 46260 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0216 02:05:37.845774 46260 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0216 02:05:37.846032 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:37.858234 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.858452 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.858502 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.858545 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.858592 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.858628 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.858658 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.858692 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.858718 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.858804 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.858841 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.916725 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:37.917268 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:37.917304 46260 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0216 02:05:37.917322 46260 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0216 02:05:37.917785 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHA (0.12s) === RUN TestCreateClusterHAGCE I0216 02:05:37.931312 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.931563 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.931615 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.931663 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.931712 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.931755 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.931785 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.931822 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.931852 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.931961 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.932002 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.971745 46260 new_cluster.go:1279] Cloud Provider ID = gce I0216 02:05:37.971888 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterHAGCE (0.05s) === RUN TestCreateClusterGCE I0216 02:05:37.980344 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:37.980535 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:37.980574 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:37.980603 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:37.980645 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:37.980677 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:37.980701 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:37.980726 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:37.980746 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:37.980820 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:37.980845 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.050479 46260 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0216 02:05:38.050542 46260 new_cluster.go:1279] Cloud Provider ID = gce I0216 02:05:38.050612 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterGCE (0.08s) === RUN TestCreateClusterHASharedZone I0216 02:05:38.056194 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.056341 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.056374 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.056403 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.056432 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.056457 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.056476 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.056498 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.056516 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.056580 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.056608 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0216 02:05:38.106105 46260 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0216 02:05:38.106151 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.106374 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.106642 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZone (0.06s) === RUN TestCreateClusterHASharedZones I0216 02:05:38.113397 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.113545 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.113574 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.113602 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.113631 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.113655 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.113673 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.113696 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.113712 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.113775 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.113799 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } W0216 02:05:38.151352 46260 new_cluster.go:887] Running with masters in the same AZs; redundancy will be reduced I0216 02:05:38.151394 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.151731 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.151746 46260 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0216 02:05:38.152003 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterHASharedZones (0.05s) === RUN TestCreateClusterPrivate I0216 02:05:38.164347 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.164539 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.164578 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.164610 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.164652 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.164680 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.164697 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.164721 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.164739 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.164808 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.164835 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.213952 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.214194 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.214212 46260 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0216 02:05:38.214552 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:38.214584 46260 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterPrivate (0.06s) === RUN TestCreateClusterPrivateGCE I0216 02:05:38.221466 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.221662 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.221699 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.221775 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.221807 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.221835 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.221852 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.221875 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.221894 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.221967 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.221992 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.256312 46260 new_cluster.go:532] VMs will be configured to use specified Service Account: test-account@testproject.iam.gserviceaccount.com I0216 02:05:38.256388 46260 new_cluster.go:1279] Cloud Provider ID = gce I0216 02:05:38.256475 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 --- PASS: TestCreateClusterPrivateGCE (0.04s) === RUN TestCreateClusterWithNGWSpecified I0216 02:05:38.263293 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.263484 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.263526 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.263556 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.263590 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.263619 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.263639 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.263665 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.263683 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.263761 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.263788 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.307290 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.307506 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.307522 46260 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0216 02:05:38.307837 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:38.307867 46260 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithNGWSpecified (0.05s) === RUN TestCreateClusterWithINGWSpecified I0216 02:05:38.314260 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.314480 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.314526 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.314577 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.314619 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.314648 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.314666 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.314690 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.314707 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.314779 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.314806 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.350367 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.350555 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.350572 46260 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0216 02:05:38.350881 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:38.350911 46260 utils.go:202] Found matching hosted zone ".private.example.com", but it was "Private" and we require "Public" --- PASS: TestCreateClusterWithINGWSpecified (0.04s) === RUN TestCreateClusterSharedVPC I0216 02:05:38.356809 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.356936 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.356961 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.356988 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.357013 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.357035 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.357053 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.357076 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.357092 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.357143 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.357167 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.384306 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.384432 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.384475 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.384504 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.384520 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.384554 46260 subnets.go:185] Assigned CIDR 10.2.0.0/15 to subnet us-test-1a I0216 02:05:38.384742 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedVPC (0.03s) === RUN TestCreateClusterSharedSubnets I0216 02:05:38.389654 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.389778 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.389805 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.389829 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.389853 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.389877 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.389894 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.389914 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.389928 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.389980 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.390003 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.434036 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.434085 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.434160 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.434295 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.434319 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.434343 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.434358 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.434659 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnets (0.05s) === RUN TestCreateClusterSharedSubnetsVpcLookup I0216 02:05:38.440723 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.440982 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.441029 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.441070 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.441114 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.441154 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.441182 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.441222 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.441246 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.441350 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.441393 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.524287 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:05:38.524377 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.524412 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.524494 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.524645 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.524666 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.524689 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.524704 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.525018 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterSharedSubnetsVpcLookup (0.09s) === RUN TestCreateClusterPrivateSharedSubnets I0216 02:05:38.530451 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.530668 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.530704 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.530732 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.530770 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.530798 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.530819 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.530846 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.530863 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.530974 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.531005 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.559374 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.559427 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.559510 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.559530 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.559557 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.559715 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.559741 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.559765 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:05:38.559780 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "vpc-id", Values: ["vpc-12345678"] }] } I0216 02:05:38.560139 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterPrivateSharedSubnets (0.04s) === RUN TestCreateClusterIPv6 I0216 02:05:38.567560 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.567951 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.568023 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.568191 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.568257 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.568285 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.568302 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.568325 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.568342 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.568427 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.568479 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.636495 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.636694 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.637244 46260 zones.go:93] ListHostedZonesPages { } --- PASS: TestCreateClusterIPv6 (0.08s) === RUN TestCreateClusterDifferentAMIs I0216 02:05:38.643461 46260 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0216 02:05:38.643477 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.643628 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.643660 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.643691 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.643726 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.643755 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.643773 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.643798 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.643818 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.643886 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.643915 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.685548 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.685599 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.685612 46260 subnets.go:199] Assigned CIDR 172.20.0.0/22 to subnet utility-us-test-1a I0216 02:05:38.685822 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:38.692508 46260 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestCreateClusterDifferentAMIs (0.05s) === RUN TestCreateClusterKarpenter I0216 02:05:38.692568 46260 featureflag.go:160] FeatureFlag "Karpenter"=true I0216 02:05:38.692580 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.692702 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.692730 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.692753 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.692777 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.692802 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.692819 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.692837 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.692850 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.692904 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.10.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.692928 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "10.11.0.0/24", VpcId: "vpc-12345678" } I0216 02:05:38.721124 46260 new_cluster.go:1279] Cloud Provider ID = aws I0216 02:05:38.721258 46260 subnets.go:185] Assigned CIDR 172.20.32.0/19 to subnet us-test-1a I0216 02:05:38.721274 46260 subnets.go:185] Assigned CIDR 172.20.64.0/19 to subnet us-test-1b I0216 02:05:38.721283 46260 subnets.go:185] Assigned CIDR 172.20.96.0/19 to subnet us-test-1c I0216 02:05:38.721509 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:38.729309 46260 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestCreateClusterKarpenter (0.04s) === RUN TestParseCloudLabels --- PASS: TestParseCloudLabels (0.00s) === RUN TestConfirmation --- PASS: TestConfirmation (0.00s) === RUN TestEditInstanceGroup I0216 02:05:38.729630 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.729797 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.729830 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.729883 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.729914 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.729941 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.729957 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.729981 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.729998 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.735371 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2021-02-05"] }], Owners: ["383156758163"] } --- PASS: TestEditInstanceGroup (0.01s) === RUN TestMinimal I0216 02:05:38.736535 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:38.736695 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:38.736722 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:38.736747 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:38.736773 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:38.736824 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:38.736843 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:38.736865 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:38.736879 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:38.751284 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:38.752964 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:38.753025 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0216 02:05:41.393993 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:44.643783 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0216 02:05:44.644266 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:44.646146 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:44.647042 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:44.649010 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:44.649035 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:44.649124 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:44.649164 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:05:44.652724 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:05:44.653276 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:44.653879 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:44.654101 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:44.655311 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:44.655387 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:05:44.655721 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:05:44.655937 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:05:44.682938 46260 target.go:112] Terraform output is in /tmp/test1017863489/out --- PASS: TestMinimal (5.95s) === RUN TestMinimal_v1_23 I0216 02:05:44.685507 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:44.685657 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:44.685696 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:44.685734 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:44.685768 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:44.685828 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:44.685846 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:44.685869 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:44.685887 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:44.703750 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0216 02:05:44.705740 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:44.705805 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.049876 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:46.166287 46260 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0216 02:05:46.166697 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:46.167261 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:46.167352 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:46.167285 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:46.167394 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:46.167547 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:46.169354 46260 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0216 02:05:46.172795 46260 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0216 02:05:46.173202 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.173470 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.174870 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.176037 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.176152 46260 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0216 02:05:46.176493 46260 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0216 02:05:46.176705 46260 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0216 02:05:46.210882 46260 target.go:112] Terraform output is in /tmp/test4226306839/out --- PASS: TestMinimal_v1_23 (1.53s) === RUN TestMinimal_v1_24 I0216 02:05:46.213654 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:46.213812 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:46.213854 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:46.213888 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:46.213921 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:46.213995 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:46.214014 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:46.214038 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:46.214057 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:46.233553 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:05:46.236842 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.236939 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.677533 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:46.817483 46260 executor.go:111] Tasks: 0 done / 93 total; 46 can run W0216 02:05:46.817847 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:46.818070 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:46.818113 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:05:46.818191 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:46.818140 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:46.818234 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:46.819577 46260 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0216 02:05:46.822524 46260 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0216 02:05:46.822781 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.822951 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.823652 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.824222 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.824287 46260 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0216 02:05:46.824482 46260 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0216 02:05:46.824604 46260 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0216 02:05:46.861464 46260 target.go:112] Terraform output is in /tmp/test1803190656/out --- PASS: TestMinimal_v1_24 (0.65s) === RUN TestMinimal_v1_25 I0216 02:05:46.866462 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:46.866780 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:46.866879 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:46.866971 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:46.867052 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:46.867183 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:46.867244 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:46.867333 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:46.867392 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:46.886645 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:05:46.888694 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:46.888760 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.111299 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.241412 46260 executor.go:111] Tasks: 0 done / 93 total; 46 can run I0216 02:05:48.241927 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:48.242129 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:48.241982 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:05:48.242215 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:48.242207 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:48.242321 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:48.244016 46260 executor.go:111] Tasks: 46 done / 93 total; 19 can run I0216 02:05:48.248571 46260 executor.go:111] Tasks: 65 done / 93 total; 24 can run I0216 02:05:48.248867 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.249099 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.250330 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.251626 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.251745 46260 executor.go:111] Tasks: 89 done / 93 total; 2 can run I0216 02:05:48.252057 46260 executor.go:111] Tasks: 91 done / 93 total; 2 can run I0216 02:05:48.252238 46260 executor.go:111] Tasks: 93 done / 93 total; 0 can run I0216 02:05:48.295656 46260 target.go:112] Terraform output is in /tmp/test2652865744/out --- PASS: TestMinimal_v1_25 (1.43s) === RUN TestMinimal_v1_26 I0216 02:05:48.299023 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.299200 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.299251 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.299304 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.299358 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.299435 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.299471 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.299515 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.299545 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.315405 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:05:48.317308 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:48.317380 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:49.657492 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:49.802074 46260 executor.go:111] Tasks: 0 done / 92 total; 45 can run W0216 02:05:49.802666 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:49.802727 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:49.802749 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:49.802845 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:49.804225 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:49.805177 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:49.807718 46260 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0216 02:05:49.811759 46260 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0216 02:05:49.812225 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:49.812426 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:49.813145 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:49.814073 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:49.814146 46260 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0216 02:05:49.814316 46260 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0216 02:05:49.814433 46260 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0216 02:05:49.850946 46260 target.go:112] Terraform output is in /tmp/test3122177/out --- PASS: TestMinimal_v1_26 (1.55s) === RUN TestHetzner I0216 02:05:49.853935 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:05:51.079981 46260 executor.go:111] Tasks: 0 done / 50 total; 39 can run I0216 02:05:51.082226 46260 executor.go:111] Tasks: 39 done / 50 total; 5 can run I0216 02:05:51.083420 46260 executor.go:111] Tasks: 44 done / 50 total; 2 can run I0216 02:05:51.088498 46260 executor.go:111] Tasks: 46 done / 50 total; 4 can run I0216 02:05:51.091342 46260 executor.go:111] Tasks: 50 done / 50 total; 0 can run I0216 02:05:51.112844 46260 target.go:112] Terraform output is in /tmp/test667001616/out --- PASS: TestHetzner (1.26s) === RUN TestNvidia I0216 02:05:51.117478 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.117749 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.117812 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.117881 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.117940 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.118042 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.118081 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.118126 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.118163 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.145112 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:51.148173 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.148267 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.223194 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:51.314596 46260 executor.go:111] Tasks: 0 done / 88 total; 44 can run W0216 02:05:51.314900 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:51.315009 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:51.315042 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:51.315131 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:51.315268 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:51.315289 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:51.317128 46260 executor.go:111] Tasks: 44 done / 88 total; 19 can run I0216 02:05:51.322261 46260 executor.go:111] Tasks: 63 done / 88 total; 21 can run I0216 02:05:51.324108 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.332302 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.334582 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.334866 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.334953 46260 executor.go:111] Tasks: 84 done / 88 total; 2 can run I0216 02:05:51.335222 46260 executor.go:111] Tasks: 86 done / 88 total; 2 can run I0216 02:05:51.335380 46260 executor.go:111] Tasks: 88 done / 88 total; 0 can run I0216 02:05:51.379459 46260 target.go:112] Terraform output is in /tmp/test1573155633/out --- PASS: TestNvidia (0.26s) === RUN TestMinimalGossip I0216 02:05:51.382135 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.382303 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.382346 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.382383 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.382416 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.382479 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.382499 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.382522 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.382539 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:51.402594 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.402663 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.554768 46260 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0216 02:05:51.555328 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:51.555348 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:51.555489 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:51.557237 46260 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0216 02:05:51.561892 46260 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0216 02:05:51.562144 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.562307 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.563208 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.563726 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.563802 46260 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0216 02:05:51.564096 46260 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0216 02:05:51.564247 46260 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0216 02:05:51.604240 46260 target.go:112] Terraform output is in /tmp/test3148592629/out --- PASS: TestMinimalGossip (0.23s) === RUN TestMinimalGossipIRSA I0216 02:05:51.607795 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.608024 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.608080 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.608139 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.608196 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.608292 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.608330 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.608373 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.608404 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:51.636596 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.636710 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.785441 46260 executor.go:111] Tasks: 0 done / 89 total; 44 can run W0216 02:05:51.785695 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:51.787041 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:51.788928 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.k8s.local-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:51.791452 46260 executor.go:111] Tasks: 44 done / 89 total; 20 can run I0216 02:05:51.796661 46260 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0216 02:05:51.796993 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.797189 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.798372 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.799625 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:51.799744 46260 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0216 02:05:51.800113 46260 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0216 02:05:51.800352 46260 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0216 02:05:51.852765 46260 target.go:112] Terraform output is in /tmp/test2406500078/out --- PASS: TestMinimalGossipIRSA (0.25s) === RUN TestMinimalGCE I0216 02:05:51.857432 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.859379 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0216 02:05:52.689089 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:52.689153 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:52.689234 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0216 02:05:52.689256 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0216 02:05:52.689270 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:05:52.689288 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:05:52.689329 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:05:52.690351 46260 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0216 02:05:52.695503 46260 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0216 02:05:52.700372 46260 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0216 02:05:52.703108 46260 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0216 02:05:52.703543 46260 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0216 02:05:52.725972 46260 target.go:112] Terraform output is in /tmp/test1076432203/out --- PASS: TestMinimalGCE (0.87s) === RUN TestMinimalGCEPrivate I0216 02:05:52.728358 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:52.729476 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:05:53.461985 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:53.462052 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:53.462132 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/main" W0216 02:05:53.462152 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com/backups/etcd/events" W0216 02:05:53.462166 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0216 02:05:53.462187 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-private.example.com" W0216 02:05:53.462226 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:05:53.463491 46260 executor.go:111] Tasks: 0 done / 66 total; 40 can run I0216 02:05:53.469051 46260 executor.go:111] Tasks: 40 done / 66 total; 18 can run I0216 02:05:53.474606 46260 executor.go:111] Tasks: 58 done / 66 total; 4 can run I0216 02:05:53.477665 46260 executor.go:111] Tasks: 62 done / 66 total; 3 can run I0216 02:05:53.478899 46260 executor.go:111] Tasks: 65 done / 66 total; 1 can run I0216 02:05:53.479112 46260 executor.go:111] Tasks: 66 done / 66 total; 0 can run I0216 02:05:53.503351 46260 target.go:112] Terraform output is in /tmp/test1928498959/out --- PASS: TestMinimalGCEPrivate (0.78s) === RUN TestMinimalGCEInternalLoadBalancer I0216 02:05:53.505933 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:53.507304 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:05:53.714982 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:53.715076 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:53.715215 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/main" W0216 02:05:53.715249 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com/backups/etcd/events" W0216 02:05:53.715273 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0216 02:05:53.715304 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-ilb.example.com" W0216 02:05:53.715368 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0216 02:05:53.715412 46260 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0216 02:05:53.717405 46260 executor.go:111] Tasks: 0 done / 68 total; 41 can run I0216 02:05:53.722524 46260 executor.go:111] Tasks: 41 done / 68 total; 17 can run W0216 02:05:53.727766 46260 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0216 02:05:53.727802 46260 executor.go:111] Tasks: 58 done / 68 total; 4 can run I0216 02:05:53.729516 46260 executor.go:111] Tasks: 62 done / 68 total; 3 can run I0216 02:05:53.730222 46260 executor.go:111] Tasks: 65 done / 68 total; 2 can run I0216 02:05:53.730535 46260 executor.go:111] Tasks: 67 done / 68 total; 1 can run I0216 02:05:53.730718 46260 executor.go:111] Tasks: 68 done / 68 total; 0 can run I0216 02:05:53.753290 46260 target.go:112] Terraform output is in /tmp/test2391275082/out --- PASS: TestMinimalGCEInternalLoadBalancer (0.25s) === RUN TestMinimalGCELongClusterName I0216 02:05:53.755711 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:53.756851 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0216 02:05:53.975784 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:53.975867 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:53.975953 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0216 02:05:53.975976 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0216 02:05:53.975991 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0216 02:05:53.976014 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0216 02:05:53.976061 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:05:53.977178 46260 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0216 02:05:53.981931 46260 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0216 02:05:53.985076 46260 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0216 02:05:53.986564 46260 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0216 02:05:53.986707 46260 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0216 02:05:54.010916 46260 target.go:112] Terraform output is in /tmp/test891967819/out --- PASS: TestMinimalGCELongClusterName (0.26s) === RUN TestMinimalGCEInternalLoadBalancerLongClusterName I0216 02:05:54.013615 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:54.014921 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0216 02:05:54.226847 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:54.226911 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:54.227009 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/main" W0216 02:05:54.227028 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com/backups/etcd/events" W0216 02:05:54.227043 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0216 02:05:54.227066 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce-with-a-very-very-very-very-very-long-name.example.com" W0216 02:05:54.227107 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0216 02:05:54.227138 46260 autoscalinggroup.go:284] Not hooking the instance group manager up to anything. I0216 02:05:54.228254 46260 executor.go:111] Tasks: 0 done / 67 total; 42 can run I0216 02:05:54.232681 46260 executor.go:111] Tasks: 42 done / 67 total; 16 can run W0216 02:05:54.235892 46260 executor.go:128] Lifecycle set to ExistsAndWarnIfChanges and object was not found. I0216 02:05:54.235915 46260 executor.go:111] Tasks: 58 done / 67 total; 5 can run I0216 02:05:54.238291 46260 executor.go:111] Tasks: 63 done / 67 total; 2 can run I0216 02:05:54.238448 46260 executor.go:111] Tasks: 65 done / 67 total; 1 can run I0216 02:05:54.238572 46260 executor.go:111] Tasks: 66 done / 67 total; 1 can run I0216 02:05:54.238690 46260 executor.go:111] Tasks: 67 done / 67 total; 0 can run I0216 02:05:54.259249 46260 target.go:112] Terraform output is in /tmp/test1265345108/out --- PASS: TestMinimalGCEInternalLoadBalancerLongClusterName (0.25s) === RUN TestHA I0216 02:05:54.261733 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:54.261894 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:54.261937 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:54.261972 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:54.262006 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:54.262070 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:54.262088 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:54.262112 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:54.262131 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:54.281467 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:54.283555 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.283616 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.283651 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.283684 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.357222 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:54.440025 46260 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0216 02:05:54.441351 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:54.441480 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:54.441542 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:54.441544 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:54.441583 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:54.441611 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:54.441782 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:54.442147 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.ha.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:54.445957 46260 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0216 02:05:54.450553 46260 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0216 02:05:54.450815 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.450958 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.451160 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.451343 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.451939 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.453556 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.454396 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.455667 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.455766 46260 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0216 02:05:54.456121 46260 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0216 02:05:54.456338 46260 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0216 02:05:54.510474 46260 target.go:112] Terraform output is in /tmp/test1661106208/out --- PASS: TestHA (0.25s) === RUN TestHighAvailabilityGCE I0216 02:05:54.513666 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:54.514917 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:05:54.749673 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:05:54.749772 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:05:54.749886 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/main" W0216 02:05:54.749914 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com/backups/etcd/events" W0216 02:05:54.749937 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0216 02:05:54.749972 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/ha-gce.example.com" W0216 02:05:54.750029 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0216 02:05:54.750103 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups W0216 02:05:54.750168 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:05:54.752281 46260 executor.go:111] Tasks: 0 done / 83 total; 48 can run I0216 02:05:54.754891 46260 executor.go:111] Tasks: 48 done / 83 total; 20 can run I0216 02:05:54.759995 46260 executor.go:111] Tasks: 68 done / 83 total; 7 can run I0216 02:05:54.762758 46260 executor.go:111] Tasks: 75 done / 83 total; 5 can run I0216 02:05:54.763894 46260 executor.go:111] Tasks: 80 done / 83 total; 3 can run I0216 02:05:54.764086 46260 executor.go:111] Tasks: 83 done / 83 total; 0 can run I0216 02:05:54.801268 46260 target.go:112] Terraform output is in /tmp/test3815861275/out --- PASS: TestHighAvailabilityGCE (0.29s) === RUN TestComplex I0216 02:05:54.806146 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:54.806369 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:54.806421 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:54.806535 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:54.806584 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:54.806624 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:54.806652 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:54.806687 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:54.806712 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:54.829024 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:05:54.831466 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:54.831532 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.034104 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:56.205514 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run W0216 02:05:56.205784 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:56.206146 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:56.206263 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:56.206290 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:56.206370 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:56.209971 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:05:56.213160 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:05:56.213455 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.213609 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.214271 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.216958 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.217062 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:05:56.217544 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:05:56.217723 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0216 02:05:56.283170 46260 target.go:112] Terraform output is in /tmp/test280621244/out I0216 02:05:56.286220 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:56.286395 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:56.286446 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:56.286482 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:56.286516 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:56.286560 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:56.286582 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:56.286614 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:56.286633 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:56.320164 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:05:56.322826 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.322898 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.397618 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:56.556811 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0216 02:05:56.557372 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:56.557400 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:56.557489 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:05:56.557511 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:56.557521 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:56.559451 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:05:56.567506 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:05:56.568045 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.568940 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.569302 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.570796 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.570898 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:05:56.571312 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:05:56.571499 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0216 02:05:56.616337 46260 target.go:112] Terraform output is in /tmp/test3009774036/out --- PASS: TestComplex (1.81s) === RUN TestCompress I0216 02:05:56.619158 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:56.619323 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:56.619367 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:56.619400 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:56.619433 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:56.619495 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:56.619513 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:56.619537 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:56.619555 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:56.633998 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:56.636900 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.637023 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.710233 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:56.787771 46260 executor.go:111] Tasks: 0 done / 86 total; 42 can run W0216 02:05:56.790704 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:56.790740 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:56.791099 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:56.791156 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:56.791312 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:56.793390 46260 executor.go:111] Tasks: 42 done / 86 total; 19 can run I0216 02:05:56.796569 46260 executor.go:111] Tasks: 61 done / 86 total; 21 can run I0216 02:05:56.796795 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.796921 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.799217 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.802237 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.802372 46260 executor.go:111] Tasks: 82 done / 86 total; 2 can run I0216 02:05:56.802949 46260 executor.go:111] Tasks: 84 done / 86 total; 2 can run I0216 02:05:56.803565 46260 executor.go:111] Tasks: 86 done / 86 total; 0 can run I0216 02:05:56.835180 46260 target.go:112] Terraform output is in /tmp/test573888135/out --- PASS: TestCompress (0.22s) === RUN TestExternalPolicies I0216 02:05:56.837594 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:56.837748 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:56.837786 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:56.837818 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:56.837852 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:56.837912 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:56.837929 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:56.837952 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:56.837972 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:56.868035 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:56.871614 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.871723 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:56.944650 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:57.038434 46260 executor.go:111] Tasks: 0 done / 100 total; 45 can run W0216 02:05:57.038680 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:57.038912 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:57.038946 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0216 02:05:57.039028 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:57.039042 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:57.039122 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externalpolicies.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:57.041202 46260 executor.go:111] Tasks: 45 done / 100 total; 20 can run I0216 02:05:57.044342 46260 executor.go:111] Tasks: 65 done / 100 total; 30 can run I0216 02:05:57.044667 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.044831 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.045514 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.046153 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.046220 46260 executor.go:111] Tasks: 95 done / 100 total; 3 can run I0216 02:05:57.046498 46260 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0216 02:05:57.046641 46260 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0216 02:05:57.107782 46260 target.go:112] Terraform output is in /tmp/test1087692375/out --- PASS: TestExternalPolicies (0.27s) === RUN TestMinimalIPv6 I0216 02:05:57.111592 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:57.111819 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:57.111871 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:57.111948 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:57.112000 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:57.112088 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:57.112115 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:57.112151 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:57.112177 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:57.142932 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:57.147510 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.147656 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.228317 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:57.388159 46260 executor.go:111] Tasks: 0 done / 106 total; 45 can run W0216 02:05:57.388633 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:57.390132 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:57.390220 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:57.392549 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:57.392590 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:57.392701 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:57.397402 46260 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0216 02:05:57.402008 46260 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0216 02:05:57.402275 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.402509 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.403182 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.404047 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.404141 46260 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0216 02:05:57.404491 46260 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0216 02:05:57.404784 46260 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0216 02:05:57.453148 46260 target.go:112] Terraform output is in /tmp/test3581481603/out --- PASS: TestMinimalIPv6 (0.35s) === RUN TestMinimalIPv6Private I0216 02:05:57.457359 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:57.457719 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:57.457776 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:57.457829 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:57.457884 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:57.457971 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:57.458000 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:57.458038 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:57.458065 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:57.488307 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0216 02:05:57.492117 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.492208 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.568167 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:57.730879 46260 executor.go:111] Tasks: 0 done / 124 total; 46 can run I0216 02:05:57.731759 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:57.731813 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:57.731923 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:05:57.732875 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:57.733192 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:57.735407 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:57.737502 46260 executor.go:111] Tasks: 46 done / 124 total; 23 can run I0216 02:05:57.741489 46260 executor.go:111] Tasks: 69 done / 124 total; 36 can run I0216 02:05:57.741973 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.742145 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.742774 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.743593 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.743660 46260 executor.go:111] Tasks: 105 done / 124 total; 11 can run I0216 02:05:57.744038 46260 executor.go:111] Tasks: 116 done / 124 total; 8 can run I0216 02:05:57.744263 46260 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0216 02:05:57.819954 46260 target.go:112] Terraform output is in /tmp/test3251640314/out --- PASS: TestMinimalIPv6Private (0.37s) === RUN TestMinimalIPv6Calico I0216 02:05:57.825436 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:57.825721 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:57.825785 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:57.825908 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:57.825966 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:57.826015 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:57.826050 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:57.826094 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:57.826127 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:57.855271 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:05:57.858759 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.858855 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:57.933993 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:58.262240 46260 executor.go:111] Tasks: 0 done / 109 total; 47 can run I0216 02:05:58.262594 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:58.262756 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.262890 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:05:58.263477 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.262921 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:58.263740 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:58.267805 46260 executor.go:111] Tasks: 47 done / 109 total; 21 can run I0216 02:05:58.271514 46260 executor.go:111] Tasks: 68 done / 109 total; 31 can run I0216 02:05:58.271971 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.272170 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.272961 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.273694 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.273768 46260 executor.go:111] Tasks: 99 done / 109 total; 5 can run I0216 02:05:58.274090 46260 executor.go:111] Tasks: 104 done / 109 total; 5 can run I0216 02:05:58.274324 46260 executor.go:111] Tasks: 109 done / 109 total; 0 can run I0216 02:05:58.314630 46260 target.go:112] Terraform output is in /tmp/test364374436/out --- PASS: TestMinimalIPv6Calico (0.49s) === RUN TestMinimalIPv6Cilium I0216 02:05:58.319813 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:58.320050 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:58.320102 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:58.320157 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:58.320211 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:58.320296 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:58.320330 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:58.320372 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:58.320400 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:58.348759 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:58.352196 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.352284 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.425203 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:58.586052 46260 executor.go:111] Tasks: 0 done / 107 total; 46 can run W0216 02:05:58.586322 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.586469 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:58.586506 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:58.586597 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:58.586582 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:58.586682 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.590739 46260 executor.go:111] Tasks: 46 done / 107 total; 21 can run I0216 02:05:58.594283 46260 executor.go:111] Tasks: 67 done / 107 total; 30 can run I0216 02:05:58.594656 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.594808 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.595337 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.596313 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.596389 46260 executor.go:111] Tasks: 97 done / 107 total; 5 can run I0216 02:05:58.596941 46260 executor.go:111] Tasks: 102 done / 107 total; 5 can run I0216 02:05:58.597269 46260 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0216 02:05:58.644421 46260 target.go:112] Terraform output is in /tmp/test2256132379/out --- PASS: TestMinimalIPv6Cilium (0.33s) === RUN TestMinimalWarmPool I0216 02:05:58.647324 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:58.647488 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:58.647531 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:58.647566 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:58.647600 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:58.647664 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:58.647684 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:58.647720 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:58.647739 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:58.663215 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:58.665285 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.665350 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.737964 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:58.858400 46260 executor.go:111] Tasks: 0 done / 89 total; 45 can run W0216 02:05:58.859406 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.859440 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:58.859486 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-warmpool.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:58.859470 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:58.859560 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:05:58.859844 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:58.864018 46260 executor.go:111] Tasks: 45 done / 89 total; 19 can run I0216 02:05:58.867090 46260 executor.go:111] Tasks: 64 done / 89 total; 21 can run I0216 02:05:58.867438 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.867611 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.868754 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.869538 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.869616 46260 executor.go:111] Tasks: 85 done / 89 total; 2 can run I0216 02:05:58.869814 46260 executor.go:111] Tasks: 87 done / 89 total; 2 can run I0216 02:05:58.869938 46260 executor.go:111] Tasks: 89 done / 89 total; 0 can run I0216 02:05:58.909284 46260 target.go:112] Terraform output is in /tmp/test2513119662/out --- PASS: TestMinimalWarmPool (0.26s) === RUN TestMinimalEtcd I0216 02:05:58.911956 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:58.912101 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:58.912138 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:58.912172 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:58.912204 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:58.912261 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:58.912278 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:58.912629 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:58.912651 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:58.928061 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:58.929879 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:58.929936 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.002036 46260 zones.go:93] ListHostedZonesPages { } W0216 02:05:59.079530 46260 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc W0216 02:05:59.079571 46260 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 10 W0216 02:05:59.080599 46260 model.go:248] overloading image in manifest (embedded etcd manifest) with images gcr.io/k8s-staging-etcdadm/etcd:v20210430-v0.1.3-739-g7da12acc I0216 02:05:59.088012 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0216 02:05:59.088530 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:05:59.088616 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.088566 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:59.088660 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:05:59.088724 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.088798 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-etcd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:59.092794 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:05:59.097385 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:05:59.097677 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.098506 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.099584 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.100555 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.100855 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:05:59.101821 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:05:59.101996 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:05:59.152223 46260 target.go:112] Terraform output is in /tmp/test2653304239/out --- PASS: TestMinimalEtcd (0.24s) === RUN TestMinimalGp3 I0216 02:05:59.155876 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:59.156117 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:59.156171 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:59.156219 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:59.156264 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:59.156353 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:59.156381 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:59.156415 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:59.156440 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:59.180876 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:59.183569 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.183653 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.256051 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:59.345716 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0216 02:05:59.346240 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.346435 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:59.346496 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:59.346460 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:59.346555 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:05:59.346583 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.347956 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:05:59.352129 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:05:59.352475 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.352687 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.353292 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.355446 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.355606 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:05:59.356267 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:05:59.356574 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:05:59.386396 46260 target.go:112] Terraform output is in /tmp/test1462469576/out --- PASS: TestMinimalGp3 (0.23s) === RUN TestMinimalLongClusterName I0216 02:05:59.388885 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:59.389038 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:59.389076 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:59.389107 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:59.389150 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:59.389209 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:59.389226 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:59.389249 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:59.389266 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:59.403665 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:59.405465 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.405523 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0216 02:05:59.477715 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:59.543807 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0216 02:05:59.544528 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:59.544533 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.544666 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:59.544682 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.this.is.truly.a.really.really.long.cluster-name.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:05:59.544692 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:59.544769 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:59.547659 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:05:59.552390 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:05:59.552663 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.552852 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.553534 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.554650 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.554737 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:05:59.554967 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:05:59.555126 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:05:59.597536 46260 target.go:112] Terraform output is in /tmp/test2561829383/out --- PASS: TestMinimalLongClusterName (0.21s) === RUN TestExistingSG I0216 02:05:59.601214 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:59.601418 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:59.601471 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:59.601516 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:59.601556 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:59.601633 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:59.601655 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:59.601684 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:59.601706 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:59.626720 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:59.629321 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.629394 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.629437 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.629480 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.703369 46260 zones.go:93] ListHostedZonesPages { } I0216 02:05:59.775303 46260 executor.go:111] Tasks: 0 done / 146 total; 53 can run W0216 02:05:59.775564 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:59.775866 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:05:59.775920 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.776086 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existingsg.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:05:59.776143 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:05:59.776151 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:05:59.776185 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:05:59.777513 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:05:59.780301 46260 executor.go:111] Tasks: 53 done / 146 total; 26 can run I0216 02:05:59.785418 46260 executor.go:111] Tasks: 79 done / 146 total; 58 can run I0216 02:05:59.785697 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.785876 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.786096 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.786328 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.788564 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.788803 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.788976 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.790754 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.790868 46260 executor.go:111] Tasks: 137 done / 146 total; 5 can run I0216 02:05:59.791163 46260 executor.go:111] Tasks: 142 done / 146 total; 4 can run I0216 02:05:59.791332 46260 executor.go:111] Tasks: 146 done / 146 total; 0 can run I0216 02:05:59.846363 46260 target.go:112] Terraform output is in /tmp/test105810580/out --- PASS: TestExistingSG (0.25s) === RUN TestBastionAdditionalUserData I0216 02:05:59.849907 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:59.850114 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:59.850169 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:59.850218 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:59.850270 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:59.850362 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:59.850392 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:59.850431 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:59.850458 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:59.868381 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:05:59.870335 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.870392 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.870427 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:05:59.942256 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:00.039668 46260 executor.go:111] Tasks: 0 done / 122 total; 48 can run W0216 02:06:00.040045 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:00.040115 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:00.040134 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:00.040121 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:00.040177 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:00.040270 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:00.040435 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.bastionuserdata.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:00.041939 46260 executor.go:111] Tasks: 48 done / 122 total; 29 can run I0216 02:06:00.047599 46260 executor.go:111] Tasks: 77 done / 122 total; 36 can run I0216 02:06:00.047932 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.048101 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.048267 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.048730 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.048793 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.050308 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.050408 46260 executor.go:111] Tasks: 113 done / 122 total; 5 can run I0216 02:06:00.050808 46260 executor.go:111] Tasks: 118 done / 122 total; 4 can run I0216 02:06:00.050980 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0216 02:06:00.095980 46260 target.go:112] Terraform output is in /tmp/test3872458234/out --- PASS: TestBastionAdditionalUserData (0.25s) === RUN TestPrivateWeave I0216 02:06:00.098893 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:00.099082 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:00.099125 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:00.099160 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:00.099193 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:00.099256 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:00.099273 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:00.099296 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:00.099314 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:00.118021 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:00.119909 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.119969 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.120006 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.192550 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:00.271876 46260 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0216 02:06:00.272333 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:00.272465 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:00.272522 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateweave.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:00.272605 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:00.272742 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:00.272762 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:00.272829 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:00.274672 46260 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0216 02:06:00.280112 46260 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0216 02:06:00.280434 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.280548 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.280593 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.280770 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.281308 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.282054 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.282128 46260 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0216 02:06:00.282603 46260 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0216 02:06:00.282787 46260 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0216 02:06:00.330562 46260 target.go:112] Terraform output is in /tmp/test2381732490/out --- PASS: TestPrivateWeave (0.24s) === RUN TestPrivateFlannel I0216 02:06:00.334540 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:00.334746 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:00.334794 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:00.334892 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:00.334938 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:00.334979 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:00.335008 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:00.335044 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:00.335070 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:00.364205 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:00.367193 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.367275 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.367346 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.439661 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:00.577542 46260 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0216 02:06:00.578072 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:00.578139 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateflannel.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:00.578178 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:00.578342 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:00.578373 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:00.578395 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:00.578498 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:00.579698 46260 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0216 02:06:00.582853 46260 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0216 02:06:00.583199 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.583981 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.584481 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.584701 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.586883 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.587675 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.587760 46260 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0216 02:06:00.588060 46260 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0216 02:06:00.588235 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:00.631580 46260 target.go:112] Terraform output is in /tmp/test5824587/out --- PASS: TestPrivateFlannel (0.30s) === RUN TestPrivateCalico I0216 02:06:00.634514 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:00.634660 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:00.634699 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:00.634762 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:00.634796 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:00.634823 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:00.634840 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:00.634867 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:00.634884 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:00.652062 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:00.654161 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.654218 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.654254 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:00.727128 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:01.034297 46260 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0216 02:06:01.034620 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:01.034663 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:01.034795 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:01.034874 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:01.035116 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:01.035140 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:01.035212 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:01.036545 46260 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0216 02:06:01.040173 46260 executor.go:111] Tasks: 78 done / 127 total; 40 can run I0216 02:06:01.040408 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.040522 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.040583 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.040761 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.042596 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.044561 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.044636 46260 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0216 02:06:01.044962 46260 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0216 02:06:01.045141 46260 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0216 02:06:01.089516 46260 target.go:112] Terraform output is in /tmp/test4220266989/out --- PASS: TestPrivateCalico (0.46s) === RUN TestPrivateCilium I0216 02:06:01.092994 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:01.093170 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:01.093213 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:01.093249 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:01.093282 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:01.093344 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:01.093362 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:01.093385 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:01.093403 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:01.110805 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:01.112919 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.112977 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.113014 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.190415 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:01.284226 46260 executor.go:111] Tasks: 0 done / 120 total; 47 can run W0216 02:06:01.284468 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:01.284803 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:01.284784 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:01.284890 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:01.284918 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:01.285002 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:01.284984 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:01.286517 46260 executor.go:111] Tasks: 47 done / 120 total; 28 can run I0216 02:06:01.289717 46260 executor.go:111] Tasks: 75 done / 120 total; 36 can run I0216 02:06:01.290076 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.290229 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.290419 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.290514 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.290916 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.291464 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.291540 46260 executor.go:111] Tasks: 111 done / 120 total; 5 can run I0216 02:06:01.291798 46260 executor.go:111] Tasks: 116 done / 120 total; 4 can run I0216 02:06:01.292001 46260 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0216 02:06:01.345467 46260 target.go:112] Terraform output is in /tmp/test1198661489/out --- PASS: TestPrivateCilium (0.26s) === RUN TestPrivateCilium2 I0216 02:06:01.348250 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:01.348413 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:01.348456 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:01.348528 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:01.348561 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:01.348590 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:01.348607 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:01.348631 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:01.348648 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:01.366704 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:01.368911 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.368975 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.369010 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.441441 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:01.923054 46260 executor.go:111] Tasks: 0 done / 124 total; 51 can run I0216 02:06:01.923657 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecilium.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:01.923711 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:01.923902 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:01.924090 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:01.924189 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:01.924121 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:01.924230 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:01.929592 46260 executor.go:111] Tasks: 51 done / 124 total; 28 can run I0216 02:06:01.935331 46260 executor.go:111] Tasks: 79 done / 124 total; 36 can run I0216 02:06:01.935668 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.935893 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.936120 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.936252 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.937329 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.938575 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:01.938686 46260 executor.go:111] Tasks: 115 done / 124 total; 5 can run I0216 02:06:01.939184 46260 executor.go:111] Tasks: 120 done / 124 total; 4 can run I0216 02:06:01.939451 46260 executor.go:111] Tasks: 124 done / 124 total; 0 can run I0216 02:06:01.996982 46260 target.go:112] Terraform output is in /tmp/test3477804733/out --- PASS: TestPrivateCilium2 (0.65s) === RUN TestPrivateCiliumAdvanced I0216 02:06:02.002401 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:02.002562 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:02.002601 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:02.002635 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:02.002668 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:02.002728 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:02.002745 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:02.002770 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:02.002790 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:02.026821 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:02.029156 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.029217 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.029252 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.102018 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:02.223217 46260 executor.go:111] Tasks: 0 done / 126 total; 53 can run W0216 02:06:02.223536 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.223503 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privateciliumadvanced.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:02.223782 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:02.225319 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.227582 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:02.227620 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:02.227713 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:02.228697 46260 executor.go:111] Tasks: 53 done / 126 total; 28 can run I0216 02:06:02.232727 46260 executor.go:111] Tasks: 81 done / 126 total; 36 can run I0216 02:06:02.233024 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.233194 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.233246 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.233397 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.234096 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.234581 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.234660 46260 executor.go:111] Tasks: 117 done / 126 total; 5 can run I0216 02:06:02.234889 46260 executor.go:111] Tasks: 122 done / 126 total; 4 can run I0216 02:06:02.235048 46260 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0216 02:06:02.299274 46260 target.go:112] Terraform output is in /tmp/test3694174747/out --- PASS: TestPrivateCiliumAdvanced (0.30s) === RUN TestPrivateCanal I0216 02:06:02.302308 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:02.302474 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:02.302518 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:02.302589 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:02.302626 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:02.302655 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:02.302674 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:02.302699 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:02.302717 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:02.319407 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:02.321396 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.321457 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.321495 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.395608 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:02.752334 46260 executor.go:111] Tasks: 0 done / 123 total; 50 can run W0216 02:06:02.753744 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.753883 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:02.753915 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:02.754024 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:02.754098 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.755593 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecanal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:02.755740 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.757848 46260 executor.go:111] Tasks: 50 done / 123 total; 28 can run I0216 02:06:02.763776 46260 executor.go:111] Tasks: 78 done / 123 total; 36 can run I0216 02:06:02.764074 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.764212 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.764276 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.764354 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.765225 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.765831 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:02.765900 46260 executor.go:111] Tasks: 114 done / 123 total; 5 can run I0216 02:06:02.766155 46260 executor.go:111] Tasks: 119 done / 123 total; 4 can run I0216 02:06:02.766338 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:02.814388 46260 target.go:112] Terraform output is in /tmp/test298660739/out --- PASS: TestPrivateCanal (0.52s) === RUN TestPrivateKopeio I0216 02:06:02.819558 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:02.819781 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:02.819837 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:02.819886 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:02.819933 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:02.820018 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:02.820047 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:02.820082 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:02.820109 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:02.841834 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:02.843789 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.843851 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.843889 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.916184 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:02.989511 46260 executor.go:111] Tasks: 0 done / 126 total; 47 can run W0216 02:06:02.989780 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:02.989886 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.990035 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:02.990060 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:02.990142 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:02.990314 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:02.990342 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:02.992025 46260 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0216 02:06:02.996934 46260 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0216 02:06:02.997239 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.997398 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.997471 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.997593 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.998258 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.998787 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:02.998860 46260 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0216 02:06:02.999214 46260 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0216 02:06:02.999384 46260 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0216 02:06:03.044770 46260 target.go:112] Terraform output is in /tmp/test3779404190/out --- PASS: TestPrivateKopeio (0.23s) === RUN TestUnmanaged I0216 02:06:03.047507 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:03.047663 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:03.047705 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:03.047737 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:03.047770 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:03.047831 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:03.047848 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:03.047873 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:03.047891 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:03.068411 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:03.070366 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.070424 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.070460 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.142140 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:03.209562 46260 executor.go:111] Tasks: 0 done / 108 total; 45 can run W0216 02:06:03.209954 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.210059 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.unmanaged.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:03.210122 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:03.210144 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.210197 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:03.210224 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:03.210306 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:03.211837 46260 executor.go:111] Tasks: 45 done / 108 total; 25 can run I0216 02:06:03.214837 46260 executor.go:111] Tasks: 70 done / 108 total; 31 can run I0216 02:06:03.215285 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.215453 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.215690 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.215787 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.216570 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.216634 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.216693 46260 executor.go:111] Tasks: 101 done / 108 total; 4 can run I0216 02:06:03.216991 46260 executor.go:111] Tasks: 105 done / 108 total; 3 can run I0216 02:06:03.217162 46260 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0216 02:06:03.256372 46260 target.go:112] Terraform output is in /tmp/test2590928/out --- PASS: TestUnmanaged (0.21s) === RUN TestPrivateSharedSubnet I0216 02:06:03.260955 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:03.261157 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:03.261201 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:03.261278 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:03.261314 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:03.261344 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:03.261364 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:03.261388 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:03.261407 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:03.278432 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:03.280316 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.280370 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.280407 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.353918 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:03.422301 46260 executor.go:111] Tasks: 0 done / 107 total; 45 can run W0216 02:06:03.422532 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.422797 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:03.422858 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:03.422869 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.422828 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:03.422911 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:03.423065 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:03.424452 46260 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0216 02:06:03.424900 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:03.427432 46260 executor.go:111] Tasks: 69 done / 107 total; 31 can run I0216 02:06:03.427699 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.427903 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.428001 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.428042 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.428471 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.429295 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.429367 46260 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0216 02:06:03.429623 46260 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0216 02:06:03.429801 46260 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0216 02:06:03.468983 46260 target.go:112] Terraform output is in /tmp/test2640328259/out --- PASS: TestPrivateSharedSubnet (0.21s) === RUN TestPrivateSharedIP I0216 02:06:03.471664 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:03.471811 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:03.471851 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:03.471881 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:03.471915 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:03.471975 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:03.471993 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:03.472016 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:03.472034 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:03.490917 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:03.492797 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.492857 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.492897 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.564752 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:03.631363 46260 executor.go:111] Tasks: 0 done / 116 total; 45 can run W0216 02:06:03.631593 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.631807 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:03.631838 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:03.631928 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:03.631940 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:03.632017 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.632082 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:03.633737 46260 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0216 02:06:03.633995 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:03.636880 46260 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0216 02:06:03.637015 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.637109 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.637267 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.637445 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.638318 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.638763 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:03.638832 46260 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0216 02:06:03.639146 46260 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0216 02:06:03.639316 46260 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0216 02:06:03.680299 46260 target.go:112] Terraform output is in /tmp/test3626354736/out --- PASS: TestPrivateSharedIP (0.21s) === RUN TestPrivateDns1 I0216 02:06:03.682981 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:03.683426 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:03.683697 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:03.683798 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:03.683856 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:03.683939 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:03.683990 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:03.684046 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:03.684084 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:03.704361 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.704425 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.704460 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.843776 46260 executor.go:111] Tasks: 0 done / 120 total; 46 can run W0216 02:06:03.844278 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:03.844419 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:03.844618 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:03.844680 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns1.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:03.846971 46260 executor.go:111] Tasks: 46 done / 120 total; 24 can run I0216 02:06:03.847696 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "internal.example.com" I0216 02:06:03.847770 46260 zones.go:40] GetHostedZone { Id: "/hostedzone/Z2AFAKE1ZON3NO" } I0216 02:06:03.847904 46260 dnszone.go:249] Existing zone "internal.example.com." found; will configure TF to reuse I0216 02:06:03.848128 46260 dnszone.go:272] No association between VPC "privatedns1.example.com" and zone "internal.example.com."; adding I0216 02:06:03.849345 46260 executor.go:111] Tasks: 70 done / 120 total; 35 can run I0216 02:06:03.849823 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.849975 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.851848 46260 executor.go:111] Tasks: 105 done / 120 total; 5 can run W0216 02:06:03.851971 46260 api.go:53] PageSize not implemented W0216 02:06:03.852065 46260 api.go:53] PageSize not implemented I0216 02:06:03.855139 46260 executor.go:111] Tasks: 110 done / 120 total; 6 can run I0216 02:06:03.855266 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.855384 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.855896 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.857506 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.857607 46260 executor.go:111] Tasks: 116 done / 120 total; 2 can run I0216 02:06:03.857868 46260 executor.go:111] Tasks: 118 done / 120 total; 2 can run I0216 02:06:03.858009 46260 executor.go:111] Tasks: 120 done / 120 total; 0 can run I0216 02:06:03.919653 46260 target.go:112] Terraform output is in /tmp/test851315355/out --- PASS: TestPrivateDns1 (0.24s) === RUN TestPrivateDns2 I0216 02:06:03.922326 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:03.922475 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:03.922511 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:03.922542 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:03.922574 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:03.922632 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:03.922650 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:03.922673 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:03.922690 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:03.941249 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.941307 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:03.941342 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.093339 46260 executor.go:111] Tasks: 0 done / 116 total; 44 can run W0216 02:06:04.093529 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:04.093619 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatedns2.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:04.094138 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:04.094236 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:04.098417 46260 executor.go:111] Tasks: 44 done / 116 total; 22 can run I0216 02:06:04.099618 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "private.example.com" I0216 02:06:04.099584 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:04.099683 46260 zones.go:40] GetHostedZone { Id: "/hostedzone/Z3AFAKE1ZOMORE" } I0216 02:06:04.099813 46260 dnszone.go:249] Existing zone "private.example.com." found; will configure TF to reuse I0216 02:06:04.099835 46260 dnszone.go:263] VPC "vpc-12345678" already associated with zone "private.example.com." I0216 02:06:04.099905 46260 executor.go:111] Tasks: 66 done / 116 total; 35 can run I0216 02:06:04.100454 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.100739 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.103947 46260 executor.go:111] Tasks: 101 done / 116 total; 5 can run W0216 02:06:04.104031 46260 api.go:53] PageSize not implemented W0216 02:06:04.104103 46260 api.go:53] PageSize not implemented I0216 02:06:04.109586 46260 executor.go:111] Tasks: 106 done / 116 total; 6 can run I0216 02:06:04.109820 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.110082 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.111175 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.112254 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.112376 46260 executor.go:111] Tasks: 112 done / 116 total; 2 can run I0216 02:06:04.112697 46260 executor.go:111] Tasks: 114 done / 116 total; 2 can run I0216 02:06:04.112884 46260 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0216 02:06:04.177380 46260 target.go:112] Terraform output is in /tmp/test4093842710/out --- PASS: TestPrivateDns2 (0.26s) === RUN TestDiscoveryFeatureGate I0216 02:06:04.182641 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:04.182861 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:04.182909 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:04.182998 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:04.183046 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:04.183099 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:04.183154 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:04.183219 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:04.183263 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:04.208459 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:04.211411 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.211489 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.288967 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:04.434733 46260 executor.go:111] Tasks: 0 done / 99 total; 51 can run W0216 02:06:04.435575 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:04.435626 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:04.435668 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:04.435684 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:04.435692 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:04.435784 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:04.441576 46260 executor.go:111] Tasks: 51 done / 99 total; 23 can run I0216 02:06:04.447542 46260 executor.go:111] Tasks: 74 done / 99 total; 21 can run I0216 02:06:04.447979 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.448251 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.449189 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.451474 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.451654 46260 executor.go:111] Tasks: 95 done / 99 total; 2 can run I0216 02:06:04.451928 46260 executor.go:111] Tasks: 97 done / 99 total; 2 can run I0216 02:06:04.452082 46260 executor.go:111] Tasks: 99 done / 99 total; 0 can run I0216 02:06:04.492641 46260 target.go:112] Terraform output is in /tmp/test3529907335/out --- PASS: TestDiscoveryFeatureGate (0.31s) === RUN TestVFSServiceAccountIssuerDiscovery I0216 02:06:04.495893 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:04.496034 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:04.496071 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:04.496102 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:04.496135 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:04.496193 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:04.496210 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:04.496236 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:04.496253 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:04.510705 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:04.512465 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.512517 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.583427 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:04.648025 46260 executor.go:111] Tasks: 0 done / 90 total; 45 can run I0216 02:06:04.648381 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:04.648545 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:04.648639 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:04.648580 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:04.648683 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:04.648780 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:04.650393 46260 executor.go:111] Tasks: 45 done / 90 total; 20 can run I0216 02:06:04.656750 46260 executor.go:111] Tasks: 65 done / 90 total; 21 can run I0216 02:06:04.656941 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.657086 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.657871 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.658422 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.658490 46260 executor.go:111] Tasks: 86 done / 90 total; 2 can run I0216 02:06:04.658732 46260 executor.go:111] Tasks: 88 done / 90 total; 2 can run I0216 02:06:04.658860 46260 executor.go:111] Tasks: 90 done / 90 total; 0 can run I0216 02:06:04.702068 46260 target.go:112] Terraform output is in /tmp/test2065111483/out --- PASS: TestVFSServiceAccountIssuerDiscovery (0.21s) === RUN TestAWSLBController I0216 02:06:04.706571 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:04.706813 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:04.706874 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:04.706925 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:04.706975 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:04.707067 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:04.707098 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:04.707135 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:04.707162 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:04.730165 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:04.733367 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.733465 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:04.806033 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:05.270367 46260 executor.go:111] Tasks: 0 done / 96 total; 49 can run W0216 02:06:05.270923 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:05.271122 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:05.271147 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:05.271150 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:05.271240 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:05.271326 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:05.278768 46260 executor.go:111] Tasks: 49 done / 96 total; 22 can run I0216 02:06:05.283509 46260 executor.go:111] Tasks: 71 done / 96 total; 21 can run I0216 02:06:05.283796 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.284019 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.284702 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.285229 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.285305 46260 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0216 02:06:05.285482 46260 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0216 02:06:05.285630 46260 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0216 02:06:05.338556 46260 target.go:112] Terraform output is in /tmp/test2517576598/out --- PASS: TestAWSLBController (0.64s) === RUN TestManyAddons I0216 02:06:05.347793 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:05.348040 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:05.348103 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:05.348214 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:05.348266 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:05.348306 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:05.348338 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:05.348379 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:05.348408 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:05.375824 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:05.379172 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.379254 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:05.451650 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:06.106991 46260 executor.go:111] Tasks: 0 done / 95 total; 50 can run W0216 02:06:06.107269 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:06.107684 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:06.107707 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:06.107767 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:06.107778 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:06.107938 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:06.110489 46260 executor.go:111] Tasks: 50 done / 95 total; 19 can run I0216 02:06:06.119668 46260 executor.go:111] Tasks: 69 done / 95 total; 22 can run I0216 02:06:06.120072 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.120277 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.121271 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.122057 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.122136 46260 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0216 02:06:06.122321 46260 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0216 02:06:06.122483 46260 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0216 02:06:06.153907 46260 target.go:112] Terraform output is in /tmp/test3787173037/out --- PASS: TestManyAddons (0.81s) === RUN TestManyAddonsCCMIRSA I0216 02:06:06.162167 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:06.162360 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:06.162405 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:06.162830 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:06.162867 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:06.162901 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:06.162919 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:06.162943 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:06.162964 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:06.179723 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:06.181909 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.181968 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.254979 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:06.973225 46260 executor.go:111] Tasks: 0 done / 111 total; 59 can run I0216 02:06:06.973612 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:06.973641 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:06.973776 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:06.973901 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:06.973923 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:06.974658 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:06.980617 46260 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0216 02:06:06.983892 46260 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0216 02:06:06.984055 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.985576 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.988540 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.990668 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:06.990773 46260 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0216 02:06:06.991115 46260 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0216 02:06:06.991312 46260 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0216 02:06:07.032685 46260 target.go:112] Terraform output is in /tmp/test3504900730/out --- PASS: TestManyAddonsCCMIRSA (0.88s) === RUN TestManyAddonsCCMIRSA23 I0216 02:06:07.038919 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:07.039065 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:07.039104 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:07.039167 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:07.039204 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:07.039233 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:07.039251 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:07.039275 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:07.039293 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:07.055734 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0216 02:06:07.057905 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.057962 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.129354 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:07.821811 46260 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0216 02:06:07.825904 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:07.826324 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:07.826927 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:07.826950 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:07.827031 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:07.824897 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:07.828633 46260 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0216 02:06:07.832956 46260 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0216 02:06:07.833126 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.834154 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.835530 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.837535 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.837616 46260 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0216 02:06:07.837827 46260 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0216 02:06:07.837955 46260 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0216 02:06:07.875589 46260 target.go:112] Terraform output is in /tmp/test3391851262/out --- PASS: TestManyAddonsCCMIRSA23 (0.85s) === RUN TestManyAddonsCCMIRSA24 I0216 02:06:07.884450 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:07.884620 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:07.884657 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:07.884725 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:07.884758 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:07.884783 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:07.884800 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:07.884823 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:07.884841 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:07.901692 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:07.903871 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.903928 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:07.975894 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:08.731975 46260 executor.go:111] Tasks: 0 done / 112 total; 60 can run W0216 02:06:08.732579 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:08.732716 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:08.732843 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:08.732887 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:08.732913 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:08.732999 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:08.737362 46260 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0216 02:06:08.744640 46260 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0216 02:06:08.745193 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.745380 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.745966 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.747963 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.748094 46260 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0216 02:06:08.748398 46260 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0216 02:06:08.748658 46260 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0216 02:06:08.811021 46260 target.go:112] Terraform output is in /tmp/test4171717373/out --- PASS: TestManyAddonsCCMIRSA24 (0.93s) === RUN TestManyAddonsCCMIRSA25 I0216 02:06:08.817450 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:08.817596 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:08.817635 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:08.817699 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:08.817734 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:08.817761 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:08.817780 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:08.817805 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:08.817823 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:08.834031 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:08.836192 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.836247 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:08.910432 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:09.562654 46260 executor.go:111] Tasks: 0 done / 112 total; 60 can run I0216 02:06:09.563276 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:09.563287 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:09.563299 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:09.563377 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:09.563460 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:09.563504 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:09.571702 46260 executor.go:111] Tasks: 60 done / 112 total; 26 can run I0216 02:06:09.575414 46260 executor.go:111] Tasks: 86 done / 112 total; 22 can run I0216 02:06:09.575726 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.575986 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.576391 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.578604 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.578739 46260 executor.go:111] Tasks: 108 done / 112 total; 2 can run I0216 02:06:09.579009 46260 executor.go:111] Tasks: 110 done / 112 total; 2 can run I0216 02:06:09.579200 46260 executor.go:111] Tasks: 112 done / 112 total; 0 can run I0216 02:06:09.638524 46260 target.go:112] Terraform output is in /tmp/test199140875/out --- PASS: TestManyAddonsCCMIRSA25 (0.83s) === RUN TestManyAddonsCCMIRSA26 I0216 02:06:09.649729 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:09.649993 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:09.650059 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:09.650168 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:09.650222 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:09.650268 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:09.650297 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:09.650338 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:09.650368 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:09.681537 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:09.684073 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.684152 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:09.759167 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:10.552769 46260 executor.go:111] Tasks: 0 done / 111 total; 59 can run I0216 02:06:10.553009 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:10.553147 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:10.553445 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:10.555650 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:10.555692 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:10.555794 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:10.558632 46260 executor.go:111] Tasks: 59 done / 111 total; 26 can run I0216 02:06:10.563385 46260 executor.go:111] Tasks: 85 done / 111 total; 22 can run I0216 02:06:10.563771 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.563986 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.565031 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.565689 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.565781 46260 executor.go:111] Tasks: 107 done / 111 total; 2 can run I0216 02:06:10.566041 46260 executor.go:111] Tasks: 109 done / 111 total; 2 can run I0216 02:06:10.566174 46260 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0216 02:06:10.606633 46260 target.go:112] Terraform output is in /tmp/test2254371943/out --- PASS: TestManyAddonsCCMIRSA26 (0.97s) === RUN TestCCM I0216 02:06:10.617405 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:10.617609 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:10.617655 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:10.617723 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:10.617757 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:10.617787 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:10.617804 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:10.617830 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:10.617849 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:10.635309 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:10.637500 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.637561 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:10.711886 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:11.327665 46260 executor.go:111] Tasks: 0 done / 96 total; 51 can run W0216 02:06:11.328000 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:11.328041 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:11.330097 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:11.330124 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:11.330206 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:11.330205 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:11.334359 46260 executor.go:111] Tasks: 51 done / 96 total; 19 can run I0216 02:06:11.337624 46260 executor.go:111] Tasks: 70 done / 96 total; 22 can run I0216 02:06:11.337931 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.338159 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.339142 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.339901 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.339973 46260 executor.go:111] Tasks: 92 done / 96 total; 2 can run I0216 02:06:11.340190 46260 executor.go:111] Tasks: 94 done / 96 total; 2 can run I0216 02:06:11.340357 46260 executor.go:111] Tasks: 96 done / 96 total; 0 can run I0216 02:06:11.375942 46260 target.go:112] Terraform output is in /tmp/test2032170215/out --- PASS: TestCCM (0.76s) === RUN TestExternalDNS I0216 02:06:11.381581 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:11.381741 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:11.381778 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:11.381809 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:11.381842 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:11.381900 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:11.381918 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:11.381943 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:11.381962 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:11.400156 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:11.403127 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.403215 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.474983 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:11.540716 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0216 02:06:11.541887 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:11.541978 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:11.543198 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:11.543223 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:11.543291 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:11.543836 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:11.546522 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:11.549588 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:11.549904 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.550064 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.550565 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.551360 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.551427 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:11.551602 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:11.551726 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:11.582782 46260 target.go:112] Terraform output is in /tmp/test477549102/out --- PASS: TestExternalDNS (0.20s) === RUN TestExternalDNSIRSA I0216 02:06:11.585380 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:11.585538 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:11.585583 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:11.585615 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:11.585648 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:11.585708 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:11.585726 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:11.585749 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:11.585767 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:11.601169 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:11.603858 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.603926 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.677551 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:11.783841 46260 executor.go:111] Tasks: 0 done / 92 total; 46 can run W0216 02:06:11.784429 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:11.784701 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:11.784780 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:11.784861 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:11.784808 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:11.784894 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:11.787104 46260 executor.go:111] Tasks: 46 done / 92 total; 21 can run I0216 02:06:11.792606 46260 executor.go:111] Tasks: 67 done / 92 total; 21 can run I0216 02:06:11.792793 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.792968 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.793639 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.794204 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.794279 46260 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0216 02:06:11.794488 46260 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0216 02:06:11.794615 46260 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0216 02:06:11.829625 46260 target.go:112] Terraform output is in /tmp/test3769562333/out --- PASS: TestExternalDNSIRSA (0.25s) === RUN TestKarpenter I0216 02:06:11.832491 46260 featureflag.go:160] FeatureFlag "Karpenter"=true I0216 02:06:11.832520 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:11.832660 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:11.832700 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:11.832764 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:11.832796 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:11.832823 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:11.832841 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:11.832864 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:11.832884 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:11.852091 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:11.854180 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.854240 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.854275 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:11.927494 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:12.053047 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.053157 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.053226 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.132100 46260 executor.go:111] Tasks: 0 done / 108 total; 53 can run W0216 02:06:12.132682 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:12.132722 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.132755 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:12.132786 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:12.132871 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:12.132852 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:12.134475 46260 executor.go:111] Tasks: 53 done / 108 total; 26 can run I0216 02:06:12.140391 46260 executor.go:111] Tasks: 79 done / 108 total; 25 can run I0216 02:06:12.140601 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.140805 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.141030 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.141232 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.141674 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.142045 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.142102 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.142694 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.142776 46260 executor.go:111] Tasks: 104 done / 108 total; 2 can run I0216 02:06:12.142957 46260 executor.go:111] Tasks: 106 done / 108 total; 2 can run I0216 02:06:12.143065 46260 executor.go:111] Tasks: 108 done / 108 total; 0 can run I0216 02:06:12.190677 46260 target.go:112] Terraform output is in /tmp/test3890828079/out I0216 02:06:12.194534 46260 featureflag.go:160] FeatureFlag "Karpenter"=false --- PASS: TestKarpenter (0.36s) === RUN TestSharedSubnet I0216 02:06:12.194628 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:12.194776 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:12.194815 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:12.194845 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:12.194877 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:12.194937 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:12.194953 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:12.194976 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:12.194997 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:12.209695 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:12.211462 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.211518 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.285535 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:12.352429 46260 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0216 02:06:12.352794 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:12.352830 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:12.352901 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:12.353023 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:12.353134 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:12.353147 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.357258 46260 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0216 02:06:12.357483 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:12.361990 46260 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0216 02:06:12.362175 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.362377 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.362835 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.363732 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.363801 46260 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0216 02:06:12.364003 46260 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0216 02:06:12.364129 46260 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0216 02:06:12.392915 46260 target.go:112] Terraform output is in /tmp/test679832145/out --- PASS: TestSharedSubnet (0.20s) === RUN TestSharedVPC I0216 02:06:12.395756 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:12.396040 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:12.396108 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:12.396177 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:12.396220 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:12.396280 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:12.396298 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:12.396320 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:12.396338 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:12.410755 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:12.412836 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.412957 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.485866 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:12.555565 46260 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0216 02:06:12.555763 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:12.556109 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.556214 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:12.556231 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:12.556294 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:12.556375 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.557525 46260 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0216 02:06:12.557691 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:12.560506 46260 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0216 02:06:12.560730 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.560925 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.561517 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.562020 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.562087 46260 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0216 02:06:12.562270 46260 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0216 02:06:12.562379 46260 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0216 02:06:12.600102 46260 target.go:112] Terraform output is in /tmp/test565166486/out --- PASS: TestSharedVPC (0.21s) === RUN TestExistingIAM I0216 02:06:12.602559 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:12.602703 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:12.602740 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:12.602772 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:12.602806 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:12.602864 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:12.602881 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:12.602904 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:12.602921 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:12.622101 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:12.624132 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.624188 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.624222 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.624254 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.698006 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:12.767430 46260 executor.go:111] Tasks: 0 done / 101 total; 51 can run W0216 02:06:12.767732 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.767703 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.existing-iam.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:12.767976 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:12.768036 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:12.767739 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:12.768110 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:12.768136 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:12.768229 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:12.769650 46260 executor.go:111] Tasks: 51 done / 101 total; 15 can run I0216 02:06:12.775702 46260 executor.go:111] Tasks: 66 done / 101 total; 27 can run I0216 02:06:12.776119 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.776240 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.776442 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.776594 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.777262 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.777568 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.777631 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.777766 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:12.777832 46260 executor.go:111] Tasks: 93 done / 101 total; 4 can run I0216 02:06:12.778107 46260 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0216 02:06:12.778315 46260 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0216 02:06:12.822676 46260 target.go:112] Terraform output is in /tmp/test1356560712/out --- PASS: TestExistingIAM (0.22s) === RUN TestPhaseNetwork I0216 02:06:12.825589 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:12.825743 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:12.825782 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:12.825814 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:12.825846 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:12.825873 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:12.825889 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:12.825918 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:12.825935 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:12.844902 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:12.846779 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.846843 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.846878 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:12.920219 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:13.018094 46260 executor.go:111] Tasks: 0 done / 119 total; 46 can run I0216 02:06:13.018525 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.lifecyclephases.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:13.018754 46260 executor.go:111] Tasks: 46 done / 119 total; 28 can run I0216 02:06:13.019253 46260 executor.go:111] Tasks: 74 done / 119 total; 36 can run I0216 02:06:13.019594 46260 executor.go:111] Tasks: 110 done / 119 total; 5 can run I0216 02:06:13.019742 46260 executor.go:111] Tasks: 115 done / 119 total; 4 can run I0216 02:06:13.019833 46260 executor.go:111] Tasks: 119 done / 119 total; 0 can run I0216 02:06:13.025087 46260 target.go:112] Terraform output is in /tmp/test2721265964/out --- PASS: TestPhaseNetwork (0.20s) === RUN TestExternalLoadBalancer I0216 02:06:13.025614 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:13.025758 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:13.025795 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:13.025826 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:13.025856 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:13.025914 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:13.025931 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:13.025956 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:13.025973 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:13.041788 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:13.043557 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.043623 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.116284 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:13.185350 46260 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0216 02:06:13.185742 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:13.186072 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:13.186152 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.186226 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:13.186253 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:13.186323 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:13.188762 46260 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0216 02:06:13.192791 46260 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0216 02:06:13.193050 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.193254 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.194822 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.197655 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.197783 46260 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0216 02:06:13.198050 46260 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0216 02:06:13.198189 46260 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0216 02:06:13.230508 46260 target.go:112] Terraform output is in /tmp/test1045440561/out --- PASS: TestExternalLoadBalancer (0.21s) === RUN TestPhaseIAM integration_test.go:816: unable to test w/o allowing failed validation --- SKIP: TestPhaseIAM (0.00s) === RUN TestPhaseCluster integration_test.go:824: unable to test w/o allowing failed validation --- SKIP: TestPhaseCluster (0.00s) === RUN TestMixedInstancesASG I0216 02:06:13.233242 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:13.233398 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:13.233440 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:13.233474 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:13.233508 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:13.233567 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:13.233584 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:13.233607 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:13.233625 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:13.255732 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:13.258034 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258094 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258127 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258158 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258191 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258218 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.258250 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.331567 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:13.418634 46260 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0216 02:06:13.419528 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.419677 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:13.419934 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.420035 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:13.420056 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:13.420133 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:13.420326 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:13.420336 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.421281 46260 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0216 02:06:13.425814 46260 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0216 02:06:13.426049 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.426236 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.426389 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.426563 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.427174 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.427574 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.427897 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.428546 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.428658 46260 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0216 02:06:13.428945 46260 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0216 02:06:13.429090 46260 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0216 02:06:13.480040 46260 target.go:112] Terraform output is in /tmp/test2758927054/out --- PASS: TestMixedInstancesASG (0.25s) === RUN TestMixedInstancesSpotASG I0216 02:06:13.483366 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:13.483511 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:13.483548 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:13.483579 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:13.483611 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:13.483669 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:13.483686 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:13.483708 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:13.483726 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:13.502998 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:13.505090 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505146 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505181 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505213 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505243 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505268 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.505295 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.576841 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:13.646388 46260 executor.go:111] Tasks: 0 done / 111 total; 53 can run W0216 02:06:13.646867 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:13.646968 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.646966 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.mixedinstances.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:13.647026 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" W0216 02:06:13.647093 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.647051 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:13.647135 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:13.647105 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.648753 46260 executor.go:111] Tasks: 53 done / 111 total; 23 can run I0216 02:06:13.656609 46260 executor.go:111] Tasks: 76 done / 111 total; 27 can run I0216 02:06:13.656821 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.657026 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.657204 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.657461 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.657946 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.658633 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.658914 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.659564 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:13.659669 46260 executor.go:111] Tasks: 103 done / 111 total; 4 can run I0216 02:06:13.659962 46260 executor.go:111] Tasks: 107 done / 111 total; 4 can run I0216 02:06:13.660168 46260 executor.go:111] Tasks: 111 done / 111 total; 0 can run I0216 02:06:13.709064 46260 target.go:112] Terraform output is in /tmp/test1604334076/out --- PASS: TestMixedInstancesSpotASG (0.23s) === RUN TestAdditionalObjects I0216 02:06:13.712177 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:13.712324 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:13.712362 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:13.712394 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:13.712427 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:13.712486 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:13.712503 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:13.712525 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:13.712542 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:13.728216 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* I0216 02:06:13.730066 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.730122 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.802510 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:13.908315 46260 executor.go:111] Tasks: 0 done / 92 total; 45 can run I0216 02:06:13.908627 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.additionalobjects.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:13.908673 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:13.908998 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:13.909138 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:13.909162 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:13.909247 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:13.910595 46260 executor.go:111] Tasks: 45 done / 92 total; 19 can run I0216 02:06:13.915077 46260 executor.go:111] Tasks: 64 done / 92 total; 24 can run I0216 02:06:13.915352 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.915555 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.916828 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.918810 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20220404"] }], Owners: ["099720109477"] } I0216 02:06:13.918973 46260 executor.go:111] Tasks: 88 done / 92 total; 2 can run I0216 02:06:13.919656 46260 executor.go:111] Tasks: 90 done / 92 total; 2 can run I0216 02:06:13.920346 46260 executor.go:111] Tasks: 92 done / 92 total; 0 can run I0216 02:06:13.957875 46260 target.go:112] Terraform output is in /tmp/test2682726998/out --- PASS: TestAdditionalObjects (0.25s) === RUN TestContainerd I0216 02:06:13.960652 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:13.960814 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:13.960859 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:13.960894 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:13.960927 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:13.960990 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:13.961007 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:13.961030 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:13.961048 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:13.978942 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:13.982057 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:13.982158 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.057003 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:14.119461 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0216 02:06:14.119901 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:14.119944 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:14.119972 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:14.120094 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:14.120660 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:14.121724 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:14.124747 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:14.128007 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:14.128330 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.128500 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.129265 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.129819 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.129883 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:14.130084 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:14.130208 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:14.170239 46260 target.go:112] Terraform output is in /tmp/test1821158386/out --- PASS: TestContainerd (0.21s) === RUN TestContainerdCustom I0216 02:06:14.172808 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:14.172981 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:14.173025 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:14.173058 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:14.173093 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:14.173158 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:14.173176 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:14.173199 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:14.173218 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:14.189461 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:14.191374 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.191439 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.263260 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:14.329184 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run W0216 02:06:14.329431 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:14.329635 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:14.329775 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:14.329824 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.containerd.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:14.329798 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:14.329886 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:14.331460 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:14.334379 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:14.334525 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.335638 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.337521 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.338880 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.338947 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:14.339171 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:14.339342 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:14.369647 46260 target.go:112] Terraform output is in /tmp/test3846856439/out --- PASS: TestContainerdCustom (0.20s) === RUN TestDockerCustom I0216 02:06:14.372173 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:14.372333 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:14.372376 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:14.372411 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:14.372444 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:14.372505 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:14.372523 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:14.372552 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:14.372571 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:14.390966 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:14.392890 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.392947 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.465953 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:14.545769 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0216 02:06:14.546086 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:14.546109 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:14.546190 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:14.548670 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:14.549200 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:14.549630 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.docker.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:14.551843 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:14.555662 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:14.555949 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.556198 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.557023 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.557626 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:14.557698 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:14.557901 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:14.558015 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:14.592700 46260 target.go:112] Terraform output is in /tmp/test807003977/out --- PASS: TestDockerCustom (0.23s) === RUN TestAPIServerNodes I0216 02:06:14.597313 46260 featureflag.go:160] FeatureFlag "APIServerNodes"=true I0216 02:06:14.597349 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:14.597584 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:14.597641 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:14.597694 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:14.597752 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:14.597855 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:14.597891 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:14.597938 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:14.597969 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:14.625944 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.22.13 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.13 ********************************************************************************* I0216 02:06:14.628630 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:14.628708 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:14.628756 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:15.868453 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:16.028604 46260 executor.go:111] Tasks: 0 done / 100 total; 47 can run W0216 02:06:16.028938 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:16.028938 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } W0216 02:06:16.029060 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:16.029206 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:16.029226 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:16.029302 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse W0216 02:06:16.029334 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:16.030764 46260 executor.go:111] Tasks: 47 done / 100 total; 24 can run I0216 02:06:16.036457 46260 executor.go:111] Tasks: 71 done / 100 total; 23 can run I0216 02:06:16.037285 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.038117 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.038334 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.039253 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.039361 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.040741 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.040855 46260 executor.go:111] Tasks: 94 done / 100 total; 3 can run I0216 02:06:16.041140 46260 executor.go:111] Tasks: 97 done / 100 total; 3 can run I0216 02:06:16.041293 46260 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0216 02:06:16.081613 46260 target.go:112] Terraform output is in /tmp/test703207182/out I0216 02:06:16.084432 46260 featureflag.go:160] FeatureFlag "APIServerNodes"=false --- PASS: TestAPIServerNodes (1.49s) === RUN TestNTHQueueProcessor I0216 02:06:16.084526 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:16.084672 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:16.084717 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:16.084752 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:16.084784 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:16.084846 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:16.084864 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:16.084886 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:16.084904 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:16.099699 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0216 02:06:16.101532 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:16.101590 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.304948 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:17.384509 46260 executor.go:111] Tasks: 0 done / 101 total; 45 can run W0216 02:06:17.384698 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:17.384698 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:17.384942 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:17.384964 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:17.385073 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:17.385222 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:17.387658 46260 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0216 02:06:17.392121 46260 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0216 02:06:17.392458 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.392654 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.393114 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.393939 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.394024 46260 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0216 02:06:17.394234 46260 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0216 02:06:17.394421 46260 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0216 02:06:17.430268 46260 target.go:112] Terraform output is in /tmp/test2320120972/out --- PASS: TestNTHQueueProcessor (1.35s) === RUN TestCustomIRSA I0216 02:06:17.433384 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.433536 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:17.433579 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:17.433613 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:17.433647 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:17.433705 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:17.433725 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:17.433748 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:17.433767 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:17.450704 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0216 02:06:17.452615 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.452674 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:17.526187 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.036999 46260 executor.go:111] Tasks: 0 done / 100 total; 50 can run W0216 02:06:18.038695 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target W0216 02:06:18.038860 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:18.039794 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:18.039831 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:18.039972 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:18.040445 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.043655 46260 executor.go:111] Tasks: 50 done / 100 total; 25 can run I0216 02:06:18.049140 46260 executor.go:111] Tasks: 75 done / 100 total; 21 can run I0216 02:06:18.050004 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.050236 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.051047 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.052093 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.052176 46260 executor.go:111] Tasks: 96 done / 100 total; 2 can run I0216 02:06:18.052409 46260 executor.go:111] Tasks: 98 done / 100 total; 2 can run I0216 02:06:18.052546 46260 executor.go:111] Tasks: 100 done / 100 total; 0 can run I0216 02:06:18.091086 46260 target.go:112] Terraform output is in /tmp/test3330183295/out --- PASS: TestCustomIRSA (0.67s) === RUN TestClusterNameDigit I0216 02:06:18.099344 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.099520 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:18.099560 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:18.099593 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:18.099626 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:18.099679 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:18.099697 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:18.099720 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:18.099737 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:18.124079 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0216 02:06:18.127566 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.127659 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.200531 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.271048 46260 executor.go:111] Tasks: 0 done / 95 total; 47 can run W0216 02:06:18.271490 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:18.271606 46260 dnszone.go:242] Check for existing route53 zone to re-use with name "" I0216 02:06:18.271638 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.123.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.271635 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0216 02:06:18.271710 46260 warmpool.go:124] ASG warm pool is not supported by the terraform target I0216 02:06:18.271717 46260 dnszone.go:249] Existing zone "example.com." found; will configure TF to reuse I0216 02:06:18.273394 46260 executor.go:111] Tasks: 47 done / 95 total; 23 can run I0216 02:06:18.276576 46260 executor.go:111] Tasks: 70 done / 95 total; 21 can run I0216 02:06:18.276906 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.277121 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.277916 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.278702 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.278775 46260 executor.go:111] Tasks: 91 done / 95 total; 2 can run I0216 02:06:18.279042 46260 executor.go:111] Tasks: 93 done / 95 total; 2 can run I0216 02:06:18.279188 46260 executor.go:111] Tasks: 95 done / 95 total; 0 can run I0216 02:06:18.331445 46260 target.go:112] Terraform output is in /tmp/test2554197057/out --- PASS: TestClusterNameDigit (0.24s) === RUN TestLifecycleMinimalAWS I0216 02:06:18.335801 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.336014 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:18.336064 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:18.336114 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:18.336165 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:18.336202 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:18.336233 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:18.336273 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:18.336304 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:18.336361 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal.example.com I0216 02:06:18.354912 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:18.356596 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.356654 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0216 02:06:18.431994 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.517549 46260 iamrole.go:89] ListRoles: { } I0216 02:06:18.519215 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0216 02:06:18.519494 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.519767 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.519937 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:18.519908 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0216 02:06:18.520041 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.520048 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:18.520067 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0216 02:06:18.520113 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.520228 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0216 02:06:18.520295 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] } I0216 02:06:18.520359 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.520572 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.520783 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.520804 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.520939 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:18.520986 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:18.521026 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:18.521084 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.521121 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:18.521287 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:18.521335 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.521539 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:18.521664 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:18.522111 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:18.522315 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0216 02:06:18.522269 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0216 02:06:18.522369 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0216 02:06:18.522383 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.522556 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0216 02:06:18.522629 46260 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0216 02:06:18.522617 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.522655 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:18.522780 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.522922 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:18.522964 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.523023 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:18.523626 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0216 02:06:18.523060 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:18.524036 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:18.523889 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:18.524116 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:18.524167 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal.example.com" }, { Key: "KubernetesCluster", Value: "minimal.example.com" }, { Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" } ] } I0216 02:06:18.524262 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:18.524421 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0216 02:06:18.524682 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:18.524806 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:18.524856 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:18.524911 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:18.525025 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:18.525177 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0216 02:06:18.525400 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:18.525524 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.525752 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:18.527321 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0216 02:06:18.530090 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:18.530323 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.530566 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-udp-1to65535-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.530843 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:18.531085 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-2382to4000-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.531293 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.531466 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.531644 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:18.531688 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:18.531733 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.533110 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-4003to65535-masters.minimal.example.com" }] }] } I0216 02:06:18.533405 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-tcp-1to2379-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.533643 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.533904 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.534027 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.534597 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.534831 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" }] }] } I0216 02:06:18.535043 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.535201 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.535419 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:18.535625 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-ingress-all-0to0-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.535958 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:18.536258 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.536406 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.536532 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.536665 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal.example.com-ingress-all-0to0-nodes.minimal.example.com" },{ Key: "KubernetesCluster", Value: "minimal.example.com" },{ Key: "kubernetes.io/cluster/minimal.example.com", Value: "owned" }] }] } I0216 02:06:18.536816 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:18.537076 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:18.537182 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:18.537222 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.537269 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:18.537343 46260 dns.go:238] Pre-creating DNS records I0216 02:06:18.537373 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:18.543415 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:18.546149 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.546242 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } ********************************************************************************* Kubelet anonymousAuth is currently turned on. This allows RBAC escalation and remote code execution possibilities. It is highly recommended you turn it off by setting 'spec.kubelet.anonymousAuth' to 'false' via 'kops edit cluster' See https://kops.sigs.k8s.io/security/#kubelet-api ********************************************************************************* I0216 02:06:18.620784 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.698332 46260 iamrole.go:89] ListRoles: { } I0216 02:06:18.700042 46260 executor.go:111] Tasks: 0 done / 87 total; 43 can run I0216 02:06:18.700998 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.701116 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:18.701228 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:18.701559 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.701786 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal.example.com"] }] } I0216 02:06:18.702034 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.702158 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:18.702195 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:18.702262 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.702404 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.705973 46260 executor.go:111] Tasks: 43 done / 87 total; 19 can run I0216 02:06:18.706170 46260 iamrole.go:161] ListAttachedRolePolicies: masters.minimal.example.com I0216 02:06:18.706112 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.706219 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal.example.com I0216 02:06:18.706263 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] },{ Name: "tag:Name", Values: ["minimal.example.com"] }] } I0216 02:06:18.706398 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.706494 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["masters.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal.example.com"] } ] } I0216 02:06:18.706667 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:18.706720 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:18.706808 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal.example.com"] } ] } I0216 02:06:18.706955 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:18.707012 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.707149 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:18.709354 46260 executor.go:111] Tasks: 62 done / 87 total; 21 can run I0216 02:06:18.709475 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.709590 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.709795 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.709946 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.710108 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.710263 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:18.710420 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:18.712489 46260 executor.go:111] Tasks: 83 done / 87 total; 2 can run I0216 02:06:18.712689 46260 executor.go:111] Tasks: 85 done / 87 total; 2 can run I0216 02:06:18.712807 46260 executor.go:111] Tasks: 87 done / 87 total; 0 can run I0216 02:06:18.715554 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:18.715628 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:18.715661 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:18.715693 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:18.715724 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:18.715759 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:18.715790 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:18.715820 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:18.715851 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:18.715884 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:18.715919 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:18.715960 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:18.715999 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:18.716030 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:18.716060 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:18.716094 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:18.716128 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:18.716161 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:18.716199 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:18.716235 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:18.716267 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:18.716297 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:18.716327 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:18.716363 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:18.716398 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:18.716431 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:18.717760 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:18.717785 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:18.717824 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.717876 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0216 02:06:18.717931 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.717989 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718037 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718093 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718138 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718178 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } I0216 02:06:18.718222 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718275 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal.example.com"] }] } I0216 02:06:18.718316 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal.example.com"] }] } W0216 02:06:18.718361 46260 tags.go:48] unknown cluster tag on vpc:vpc-12345678: "kubernetes.io/cluster/minimal.example.com"="" W0216 02:06:18.718392 46260 api.go:53] PageSize not implemented I0216 02:06:18.718411 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:18.718455 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:18.718474 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:18.718500 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:18.718518 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:18.718569 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:18.718599 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:18.718649 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:18.718700 46260 iamrole.go:89] ListRoles: { } I0216 02:06:18.718758 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:18.718803 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:18.718835 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } autoscaling-config:lt-2 ok autoscaling-group:nodes.minimal.example.com ok autoscaling-group:master-us-test-1a.masters.minimal.example.com ok I0216 02:06:18.719426 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } I0216 02:06:18.719446 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } keypair:key-1 ok I0216 02:06:18.719519 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal.example.com" } I0216 02:06:18.719513 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } iam-instance-profile:nodes.minimal.example.com ok volume:vol-1 ok autoscaling-config:lt-1 ok I0216 02:06:18.719470 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:18.719575 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } route53-record:Z1AFAKE1ZON3YO/A/api.internal.minimal.example.com. ok I0216 02:06:18.719562 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0216 02:06:18.719626 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0216 02:06:18.719638 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal.example.com" } I0216 02:06:18.719650 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } iam-instance-profile:masters.minimal.example.com ok volume:vol-2 ok I0216 02:06:18.719676 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:18.719715 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0216 02:06:18.719867 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:18.719881 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:18.719908 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:18.719936 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0216 02:06:18.719998 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:18.720006 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:18.720049 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:18.720077 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal.example.com" } I0216 02:06:18.720134 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal.example.com I0216 02:06:18.720144 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal.example.com", RoleName: "nodes.minimal.example.com" } I0216 02:06:18.720168 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal.example.com" } iam-role:nodes.minimal.example.com ok I0216 02:06:18.720189 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal.example.com" } I0216 02:06:18.720213 46260 iamrole.go:180] ListAttachedRolePolicies: masters.minimal.example.com I0216 02:06:18.720220 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal.example.com", RoleName: "masters.minimal.example.com" } I0216 02:06:18.720239 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal.example.com" } iam-role:masters.minimal.example.com ok I0216 02:06:18.720273 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:18.720316 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:18.720366 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:18.720571 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleMinimalAWS (0.38s) === RUN TestLifecycleMinimalOpenstack I0216 02:06:18.720809 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster minimal-openstack.k8s.local I0216 02:06:18.726665 46260 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:06:18.930046 46260 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0216 02:06:18.930105 46260 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0216 02:06:18.945812 46260 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0216 02:06:18.952496 46260 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0216 02:06:19.002867 46260 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0216 02:06:19.005547 46260 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0216 02:06:19.007850 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:19.007940 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:19.008055 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0216 02:06:19.012044 46260 executor.go:111] Tasks: 96 done / 96 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:06:19.215595 46260 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0216 02:06:19.215648 46260 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0216 02:06:19.228787 46260 executor.go:111] Tasks: 0 done / 96 total; 42 can run I0216 02:06:19.231354 46260 executor.go:111] Tasks: 42 done / 96 total; 45 can run I0216 02:06:19.245678 46260 executor.go:111] Tasks: 87 done / 96 total; 5 can run I0216 02:06:19.247585 46260 executor.go:111] Tasks: 92 done / 96 total; 4 can run W0216 02:06:19.249597 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:19.250425 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:19.251159 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0216 02:06:19.252391 46260 executor.go:111] Tasks: 96 done / 96 total; 0 can run SecurityGroup:504bbecc-9603-4726-a83a-5119075c7b10 ok SecurityGroup:a533eb0f-11e1-477b-948b-c5ebfb0be2ee ok Instance:f43d3426-2244-42b2-97d2-860089c94410 ok Port:4bc53f5e-9b45-4235-a3b6-956444dd084e ok Subnet:988b0481-58d3-4572-9d3f-24853e9d52c7 ok Network:190956af-32e5-4cce-8962-8660b7fc1a88 ok ServerGroup:329853e8-c663-4af6-939e-e07bba931198 ok Instance:331c889c-b265-4139-8cd8-942a0febd0ce ok SecurityGroup:431529ec-34e1-484c-b7c6-91d43b95eab0 ok Port:f7df84b2-003a-40c2-b84d-e84197140620 ok ServerGroup:7d05b03f-d428-4c94-9ee3-9efe8e6a23f6 ok SSHKey:kubernetes-minimal-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Instance:1aa95564-4c67-4c82-9619-f43b2686983e ok Port:0229ee5e-bcee-4b30-8845-b26618ffb6a0 ok --- PASS: TestLifecycleMinimalOpenstack (1.64s) === RUN TestLifecycleMinimalGCE I0216 02:06:20.356347 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:486: running lifecycle test for cluster minimal-gce.example.com I0216 02:06:20.357526 46260 network.go:228] assigned 10.0.16.0/20 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0216 02:06:20.624352 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:06:20.624457 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:06:20.624571 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:06:20.624631 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0216 02:06:20.624658 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0216 02:06:20.624684 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:06:20.624756 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:06:20.626581 46260 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0216 02:06:20.628713 46260 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0216 02:06:20.636223 46260 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0216 02:06:20.638668 46260 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0216 02:06:20.638844 46260 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0216 02:06:20.638921 46260 dns.go:238] Pre-creating DNS records ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.23.10 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.23.10 ********************************************************************************* W0216 02:06:20.858814 46260 external_access.go:36] TODO: Harmonize gcemodel ExternalAccessModelBuilder with awsmodel W0216 02:06:20.858880 46260 firewall.go:41] TODO: Harmonize gcemodel with awsmodel for firewall - GCE model is way too open W0216 02:06:20.858947 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:06:20.858976 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/main" W0216 02:06:20.858990 46260 storageacl.go:153] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com/backups/etcd/events" W0216 02:06:20.859003 46260 storageacl.go:189] unknown path, can't apply IAM policy: "memfs://tests/minimal-gce.example.com" W0216 02:06:20.859048 46260 autoscalinggroup.go:122] enabling storage-rw for etcd backups I0216 02:06:20.860153 46260 executor.go:111] Tasks: 0 done / 64 total; 41 can run I0216 02:06:20.862243 46260 executor.go:111] Tasks: 41 done / 64 total; 17 can run I0216 02:06:20.865196 46260 executor.go:111] Tasks: 58 done / 64 total; 4 can run I0216 02:06:20.869241 46260 executor.go:111] Tasks: 62 done / 64 total; 2 can run I0216 02:06:20.869371 46260 executor.go:111] Tasks: 64 done / 64 total; 0 can run I0216 02:06:20.871127 46260 gce.go:98] Scanning zones: [us-test1-a] I0216 02:06:20.871535 46260 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" I0216 02:06:20.871561 46260 gce.go:1070] backend services not found, assuming none exist in project: "testproject" region: "us-test1" FirewallRule:kubernetes-master-https-ipv6-minimal-gce-example-com ok FirewallRule:master-to-master-minimal-gce-example-com ok FirewallRule:ssh-external-to-node-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-nodes-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-minimal-gce-example-com ok FirewallRule:node-to-node-minimal-gce-example-com ok FirewallRule:ssh-external-to-master-ipv6-minimal-gce-example-com ok InstanceGroupManager:us-test1-a/a-master-us-test1-a-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/node-minimal-gce-example-com@testproject.iam.gserviceaccount.com ok FirewallRule:ssh-external-to-node-ipv6-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-ipv6-minimal-gce-example-com ok ServiceAccount:projects/testproject/serviceAccounts/control-plane-minimal-g-fu1mg6@testproject.iam.gserviceaccount.com ok FirewallRule:master-to-node-minimal-gce-example-com ok FirewallRule:kubernetes-master-https-minimal-gce-example-com ok Disk:d1-etcd-main-minimal-gce-example-com ok Disk:d1-etcd-events-minimal-gce-example-com ok FirewallRule:nodeport-external-to-node-minimal-gce-example-com ok FirewallRule:node-to-master-minimal-gce-example-com ok InstanceTemplate:nodes-minimal-gce-example-com-1676513180 ok InstanceTemplate:master-us-test1-a-minimal-do16cp-1676513180 ok Subnet:us-test1-minimal-gce-example-com ok Network:minimal-gce-example-com ok --- PASS: TestLifecycleMinimalGCE (0.52s) === RUN TestLifecycleFloatingIPOpenstack I0216 02:06:20.872733 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false lifecycle_integration_test.go:433: running lifecycle test for cluster floatingip-openstack.k8s.local I0216 02:06:20.877773 46260 subnets.go:185] Assigned CIDR 192.168.32.0/19 to subnet us-test1 ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:06:21.082400 46260 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0216 02:06:21.082455 46260 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0216 02:06:21.096325 46260 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0216 02:06:21.103649 46260 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0216 02:06:21.139177 46260 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0216 02:06:21.141278 46260 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0216 02:06:21.142753 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:21.143022 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:21.143102 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0216 02:06:21.145881 46260 executor.go:111] Tasks: 98 done / 98 total; 0 can run ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* W0216 02:06:21.343044 46260 pruning.go:112] manifest includes an object of GroupKind CSIDriver.storage.k8s.io, which will not be pruned W0216 02:06:21.343081 46260 pruning.go:112] manifest includes an object of GroupKind StorageClass.storage.k8s.io, which will not be pruned I0216 02:06:21.354533 46260 executor.go:111] Tasks: 0 done / 98 total; 43 can run I0216 02:06:21.357034 46260 executor.go:111] Tasks: 43 done / 98 total; 45 can run I0216 02:06:21.365721 46260 executor.go:111] Tasks: 88 done / 98 total; 6 can run I0216 02:06:21.366926 46260 executor.go:111] Tasks: 94 done / 98 total; 4 can run W0216 02:06:21.368598 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. W0216 02:06:21.368740 46260 bootstrapscript.go:167] exporting username and password. Consider using application credentials instead. I0216 02:06:21.370090 46260 executor.go:111] Tasks: 98 done / 98 total; 0 can run SecurityGroup:0c9aa8f1-1be8-4219-98f1-cb7e124f0f2c ok Port:ce0bd2e7-9b8c-4264-84bc-5b5f0f1c59e1 ok SSHKey:kubernetes-floatingip-openstack-k8s-local-c4_a6_ed_9a_a8_89_b9_e2_c3_9c_d6_63_eb_9c_71_57 ok Port:d4cbacf0-cd53-471b-a74b-17f7d1ad5707 ok SecurityGroup:364e2fc0-8d0e-489a-900f-377f8f97d4f4 ok Port:4008d6ca-6650-4205-a4ed-a8d6f69f79f4 ok Instance:fe1d1427-785f-4155-b154-a33793edf423 ok Router-IF:f1c66758-4fa9-49fe-ad2e-7118b8f00507 ok Router:451ea280-9749-4618-9610-4c511cbde574 ok Instance:4113502d-86bb-48f5-a373-c3df8d2232ed ok Instance:5d06f974-eac0-4aa0-95e7-d60799a10915 ok Network:d9f8883e-e2e0-436f-bff6-a72e15979ec9 ok ServerGroup:629a0687-be1c-41ed-af28-6b37af33fc66 ok Port:b61a7541-b7fe-46c9-9b3d-5859e27db361 ok ServerGroup:b8fecfac-6481-4f2a-abab-e2af0ea99097 ok Subnet:f1c66758-4fa9-49fe-ad2e-7118b8f00507 ok SecurityGroup:7ac435a7-7fc8-4caa-995d-d2b51d2ccfc5 ok --- PASS: TestLifecycleFloatingIPOpenstack (1.61s) === RUN TestLifecyclePrivateCalico I0216 02:06:22.478209 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:22.478426 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:22.478470 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:22.478504 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:22.478540 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:22.478570 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:22.478586 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:22.478610 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:22.478626 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:22.478671 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatecalico.example.com I0216 02:06:22.501371 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:22.504979 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.505075 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.505135 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.578337 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:22.924069 46260 iamrole.go:89] ListRoles: { } I0216 02:06:22.926425 46260 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0216 02:06:22.926640 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.926865 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:22.926925 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:22.926958 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } I0216 02:06:22.927045 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0216 02:06:22.927000 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:22.927122 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } I0216 02:06:22.927206 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "masters.privatecalico.example.com" }] } I0216 02:06:22.927200 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.928196 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "privatecalico.example.com" }] }] } I0216 02:06:22.928219 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.928424 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.928584 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:22.928640 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.928766 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:22.928808 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:22.928847 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:22.928985 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:22.929029 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:22.929087 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:22.929182 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } I0216 02:06:22.929257 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.929316 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatecalico.example.com" }] } I0216 02:06:22.929427 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:22.929651 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:22.932151 46260 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0216 02:06:22.932269 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:22.932371 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.932394 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0216 02:06:22.932448 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:22.932469 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0216 02:06:22.932525 46260 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0216 02:06:22.932535 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0216 02:06:22.932504 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.932550 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0216 02:06:22.932612 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0216 02:06:22.932598 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0216 02:06:22.932679 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0216 02:06:22.932706 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0216 02:06:22.932741 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0216 02:06:22.932788 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:22.932821 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] } I0216 02:06:22.932870 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:22.932859 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0216 02:06:22.932905 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0216 02:06:22.932928 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0216 02:06:22.933027 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:22.933082 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:22.933114 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:22.933139 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.933259 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0216 02:06:22.933267 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.933386 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:22.933458 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0216 02:06:22.933520 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0216 02:06:22.933583 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:22.933640 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0216 02:06:22.933698 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:22.933747 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:22.933801 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:22.933828 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:22.933856 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:22.933882 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:22.933902 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:22.933928 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:22.933948 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:22.933778 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatecalico.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatecalico.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:ModifyNetworkInterfaceAttribute\",\n \"ecr:BatchCheckLayerAvailability\",\n \"ecr:BatchGetImage\",\n \"ecr:DescribeRepositories\",\n \"ecr:GetAuthorizationToken\",\n \"ecr:GetDownloadUrlForLayer\",\n \"ecr:GetRepositoryPolicy\",\n \"ecr:ListImages\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatecalico.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } I0216 02:06:22.934020 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:22.934059 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:22.934085 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:22.934136 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:22.934169 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:22.934192 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.934244 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:22.934272 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatecalico.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:22.934319 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:22.934343 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:22.934363 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:22.934392 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatecalico.example.com" }, { Key: "KubernetesCluster", Value: "privatecalico.example.com" }, { Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0216 02:06:22.934437 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:22.935338 46260 executor.go:111] Tasks: 78 done / 127 total; 40 can run W0216 02:06:22.935532 46260 api.go:53] PageSize not implemented I0216 02:06:22.935487 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } W0216 02:06:22.935598 46260 api.go:53] PageSize not implemented I0216 02:06:22.935603 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:22.935643 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0216 02:06:22.935665 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:22.935729 46260 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.935742 46260 tags.go:74] AddTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"], Tags: [{ Key: "Name", Value: "api.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0216 02:06:22.935818 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0216 02:06:22.935845 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0216 02:06:22.935892 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatecalico-example-0uch4k" } I0216 02:06:22.935859 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0216 02:06:22.935946 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0216 02:06:22.935960 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.935991 46260 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"], Tags: [{ Key: "Name", Value: "bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] } I0216 02:06:22.936051 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0216 02:06:22.936056 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.936071 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0216 02:06:22.936097 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0216 02:06:22.936135 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-2382to4000-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.936416 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.936656 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" }] }] } I0216 02:06:22.936818 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:22.936909 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:22.936980 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-udp-1to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.937145 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:22.937217 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0216 02:06:22.937377 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.937526 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.937673 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0216 02:06:22.937818 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:22.937902 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.938100 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.938238 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.938379 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0216 02:06:22.938421 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.938573 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:22.938702 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0216 02:06:22.938837 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0216 02:06:22.938972 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatecalico.example.com-ingress-tcp-22to22-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.939117 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-1to2379-masters.privatecalico.example.com" }] }] } I0216 02:06:22.939346 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:22.939435 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.939577 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0216 02:06:22.939734 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:22.939778 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-tcp-4003to65535-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.939924 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.940006 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-bastion-elb.privatecalico.example.com" }] }] } I0216 02:06:22.940135 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatecalico.example.com-ingress-all-0to0-nodes.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.940271 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatecalico.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.940407 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:22.940454 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.940602 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatecalico.example.com-ingress-tcp-22to22-bastion.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.940772 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatecalico.example.com-ingress-4-0to0-masters.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" },{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" }] }] } I0216 02:06:22.940948 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-2" } I0216 02:06:22.940994 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.941045 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0216 02:06:22.941083 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.942194 46260 executor.go:111] Tasks: 118 done / 127 total; 5 can run I0216 02:06:22.942341 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:22.942407 46260 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:22.942336 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:22.942508 46260 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "kubernetes.io/cluster/privatecalico.example.com", Value: "owned" },{ Key: "Name", Value: "us-test-1a.privatecalico.example.com" },{ Key: "KubernetesCluster", Value: "privatecalico.example.com" }] }] } I0216 02:06:22.942565 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:22.942591 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:22.942568 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:22.942616 46260 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0216 02:06:22.942660 46260 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0216 02:06:22.942736 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:22.942825 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0216 02:06:22.942866 46260 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0216 02:06:22.942893 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:22.942915 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:22.942947 46260 dns.go:238] Pre-creating DNS records I0216 02:06:22.942961 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:22.946786 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* I0216 02:06:22.949850 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.949936 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:22.949997 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.022563 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.354111 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.355470 46260 executor.go:111] Tasks: 0 done / 127 total; 50 can run I0216 02:06:23.355631 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:23.355749 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatecalico.example.com"] }] } I0216 02:06:23.355951 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.356072 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.356151 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatecalico.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.356248 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.356365 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.356427 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:23.356445 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:23.356473 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.357636 46260 executor.go:111] Tasks: 50 done / 127 total; 28 can run I0216 02:06:23.357793 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.privatecalico.example.com I0216 02:06:23.357844 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.privatecalico.example.com I0216 02:06:23.357772 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "tag:Name", Values: ["bastion.privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatecalico.example.com"] } ] } I0216 02:06:23.357901 46260 iamrole.go:161] ListAttachedRolePolicies: masters.privatecalico.example.com I0216 02:06:23.357919 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.358057 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatecalico.example.com"] } ] } I0216 02:06:23.358168 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:23.358205 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.358293 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:23.358321 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.358393 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatecalico.example.com"] } ] } I0216 02:06:23.358482 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.358513 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.358590 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.358637 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1a.privatecalico.example.com"] }] } I0216 02:06:23.358709 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:23.358731 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:23.358767 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatecalico.example.com"] } ] } I0216 02:06:23.358859 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:23.358893 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatecalico.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.358964 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatecalico.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatecalico.example.com"] } ] } I0216 02:06:23.359044 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.360952 46260 executor.go:111] Tasks: 78 done / 127 total; 40 can run W0216 02:06:23.361117 46260 api.go:53] PageSize not implemented I0216 02:06:23.361139 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.361205 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.361229 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } I0216 02:06:23.361086 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0216 02:06:23.361282 46260 api.go:53] PageSize not implemented I0216 02:06:23.361293 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.361292 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.361336 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0216 02:06:23.361400 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatecalico-example-0uch4k" } I0216 02:06:23.361391 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.361462 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.361551 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.361752 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.361830 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.362167 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.362220 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:23.362249 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0216 02:06:23.362273 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0216 02:06:23.362310 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.362358 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.362445 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.363118 46260 executor.go:111] Tasks: 118 done / 127 total; 5 can run W0216 02:06:23.363188 46260 api.go:53] PageSize not implemented I0216 02:06:23.363203 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.363203 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.363284 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } W0216 02:06:23.363319 46260 api.go:53] PageSize not implemented I0216 02:06:23.363343 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.363290 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.363444 46260 dnsname.go:121] AliasTarget for "api.privatecalico.example.com" is "api-privatecalico-example-0uch4k.elb.cloudmock.com" I0216 02:06:23.363468 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k"] } I0216 02:06:23.363547 46260 executor.go:111] Tasks: 123 done / 127 total; 4 can run I0216 02:06:23.363648 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.363737 46260 executor.go:111] Tasks: 127 done / 127 total; 0 can run I0216 02:06:23.366872 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:23.366925 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:23.366964 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:23.367001 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:23.367036 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:23.367075 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0216 02:06:23.367111 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:23.367145 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:23.367178 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:23.367210 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:23.367242 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:23.367274 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:23.367324 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:23.367359 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0216 02:06:23.367392 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:23.367421 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:23.367447 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:23.367473 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:23.367502 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:23.367528 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:23.367557 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:23.367585 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:23.367621 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:23.367660 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:23.367697 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:23.367735 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.367771 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:23.367803 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:23.367841 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:23.367873 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:23.367904 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:23.367941 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:23.367980 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:23.368018 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:23.369449 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:23.369471 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:23.369506 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369559 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0216 02:06:23.369622 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369673 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369713 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369757 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369793 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369830 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } I0216 02:06:23.369880 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.369931 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:23.369957 46260 natgateway.go:117] DescribeNatGateways: { } I0216 02:06:23.369988 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatecalico.example.com"] }] } I0216 02:06:23.370027 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatecalico.example.com"] }] } W0216 02:06:23.370075 46260 api.go:53] PageSize not implemented I0216 02:06:23.370089 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatecalico-example-0uch4k","bastion-privatecalico-exa-hocohm"] } I0216 02:06:23.370137 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:23.370164 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:23.370180 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:23.370208 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:23.370224 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:23.370269 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.370297 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.370338 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:23.370380 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.370427 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:23.370475 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:23.370514 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0216 02:06:23.370555 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:23.370586 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:bastion.privatecalico.example.com ok I0216 02:06:23.371184 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-3 ok I0216 02:06:23.371424 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0216 02:06:23.371501 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-group:nodes.privatecalico.example.com ok I0216 02:06:23.371296 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatecalico-example-0uch4k.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatecalico.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatecalico.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.371581 46260 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } route53-record:Z1AFAKE1ZON3YO/A/api.privatecalico.example.com ok nat-gateway:nat-1 ok I0216 02:06:23.371739 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.privatecalico.example.com ok I0216 02:06:23.371944 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatecalico-example-0uch4k" } load-balancer:api-privatecalico-example-0uch4k ok I0216 02:06:23.371895 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "4", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0216 02:06:23.372168 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.372168 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0216 02:06:23.372232 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatecalico.example.com" } iam-instance-profile:bastions.privatecalico.example.com ok I0216 02:06:23.372218 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } I0216 02:06:23.372279 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } security-group:sg-1 ok I0216 02:06:23.372110 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatecalico-exa-hocohm" } load-balancer:bastion-privatecalico-exa-hocohm ok I0216 02:06:23.372396 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatecalico.example.com" } iam-instance-profile:masters.privatecalico.example.com ok I0216 02:06:23.372380 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:23.372487 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0216 02:06:23.372535 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatecalico.example.com" } iam-instance-profile:nodes.privatecalico.example.com ok I0216 02:06:23.372548 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0216 02:06:23.372627 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.372638 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok autoscaling-config:lt-2 ok I0216 02:06:23.372694 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:23.372738 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0216 02:06:23.372773 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.372803 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0216 02:06:23.372886 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.372893 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0216 02:06:23.372945 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0216 02:06:23.372974 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.372984 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatecalico.example.com" } I0216 02:06:23.373043 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.privatecalico.example.com I0216 02:06:23.373029 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:23.373060 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatecalico.example.com", RoleName: "nodes.privatecalico.example.com" } I0216 02:06:23.373094 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatecalico.example.com" } iam-role:nodes.privatecalico.example.com ok I0216 02:06:23.373083 46260 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } elastic-ip:eipalloc-1 ok I0216 02:06:23.373144 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatecalico.example.com" } I0216 02:06:23.373193 46260 iamrole.go:180] ListAttachedRolePolicies: masters.privatecalico.example.com I0216 02:06:23.373136 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }] } I0216 02:06:23.373203 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatecalico.example.com", RoleName: "masters.privatecalico.example.com" } W0216 02:06:23.373217 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.373223 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.privatecalico.example.com" } I0216 02:06:23.373229 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } iam-role:masters.privatecalico.example.com ok security-group:sg-2 ok I0216 02:06:23.373247 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:23.373257 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatecalico.example.com" } I0216 02:06:23.373290 46260 iamrole.go:180] ListAttachedRolePolicies: bastions.privatecalico.example.com I0216 02:06:23.373298 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatecalico.example.com", RoleName: "bastions.privatecalico.example.com" } I0216 02:06:23.373315 46260 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatecalico.example.com" } iam-role:bastions.privatecalico.example.com ok I0216 02:06:23.373274 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0216 02:06:23.373383 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.373395 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0216 02:06:23.373457 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:23.373510 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0216 02:06:23.373556 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:23.373618 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:23.373843 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateCalico (0.90s) === RUN TestLifecyclePrivateKopeio I0216 02:06:23.374129 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:23.374338 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:23.374371 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:23.374401 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:23.374441 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:23.374471 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:23.374497 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:23.374527 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:23.374544 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:23.374578 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster privatekopeio.example.com I0216 02:06:23.396184 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:23.399462 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.399534 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.399573 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.473473 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.572121 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.573743 46260 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0216 02:06:23.574052 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:23.574118 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.575036 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.575243 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:23.575374 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:23.575428 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.575580 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.575648 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.575864 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:23.576057 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } I0216 02:06:23.576157 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:23.576169 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0216 02:06:23.576230 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.576365 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:23.576353 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } I0216 02:06:23.576466 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.576596 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0216 02:06:23.576727 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.576803 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } I0216 02:06:23.576919 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.privatekopeio.example.com" }] } I0216 02:06:23.577035 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.577459 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:23.577496 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:23.577527 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:23.577570 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.579097 46260 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0216 02:06:23.579212 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.579302 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0216 02:06:23.579279 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.579340 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0216 02:06:23.579421 46260 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0216 02:06:23.579372 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.579465 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0216 02:06:23.579544 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0216 02:06:23.579536 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:23.579566 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0216 02:06:23.579573 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.579632 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0216 02:06:23.579679 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "privatekopeio.example.com" }] }] } I0216 02:06:23.579736 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:23.579802 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0216 02:06:23.579830 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:23.579860 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.580066 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0216 02:06:23.580079 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0216 02:06:23.580288 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0216 02:06:23.580488 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.580656 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1b"] }] } I0216 02:06:23.580746 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.580819 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] }], VpcId: "vpc-1" } I0216 02:06:23.580881 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.580690 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/privatekopeio.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/privatekopeio.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"privatekopeio.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0216 02:06:23.580968 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0216 02:06:23.580984 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.581054 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1b", CidrBlock: "172.20.8.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "Name", Value: "utility-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" } ] }], VpcId: "vpc-1" } I0216 02:06:23.581065 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0216 02:06:23.581193 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:23.581234 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0216 02:06:23.581329 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.581413 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0216 02:06:23.581485 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.privatekopeio.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:23.581535 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0216 02:06:23.581625 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0216 02:06:23.581708 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:23.581747 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:23.581794 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.581821 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "Name", Value: "private-us-test-1b.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1b" } ] } I0216 02:06:23.581879 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0216 02:06:23.581964 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-1" } I0216 02:06:23.582014 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0216 02:06:23.582040 46260 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [ { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.privatekopeio.example.com" } ] } I0216 02:06:23.582082 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0216 02:06:23.582109 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:23.582131 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:23.582183 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:23.582211 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:23.582262 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.582292 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:23.582314 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:23.582348 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "privatekopeio.example.com" }, { Key: "KubernetesCluster", Value: "privatekopeio.example.com" }, { Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0216 02:06:23.582398 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0216 02:06:23.582426 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0216 02:06:23.582446 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:23.582511 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:23.582538 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:23.582560 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:23.582590 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:23.582619 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:23.582644 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:23.582663 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.privatekopeio.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api-elb.privatekopeio.example.com" }] }], VpcId: "vpc-1" } I0216 02:06:23.582710 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:23.582734 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:23.582755 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.583785 46260 executor.go:111] Tasks: 78 done / 126 total; 39 can run I0216 02:06:23.583867 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0216 02:06:23.583911 46260 api.go:53] PageSize not implemented I0216 02:06:23.583989 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:23.584041 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0216 02:06:23.584119 46260 tags.go:74] AddTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"], Tags: [{ Key: "Name", Value: "bastion.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] } I0216 02:06:23.584041 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-udp-1to65535-masters.privatekopeio.example.com" }] }] } I0216 02:06:23.584164 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0216 02:06:23.584183 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0216 02:06:23.584221 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } W0216 02:06:23.584270 46260 api.go:53] PageSize not implemented I0216 02:06:23.584281 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0216 02:06:23.584255 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.584393 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.584417 46260 tags.go:74] AddTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"], Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "api.privatekopeio.example.com" }] } I0216 02:06:23.584382 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0216 02:06:23.584449 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.584472 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0216 02:06:23.584499 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-privatekopeio-example-tl2bv8" } I0216 02:06:23.584536 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:23.584717 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:23.584895 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0216 02:06:23.585032 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0216 02:06:23.585226 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.585332 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0216 02:06:23.585411 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0216 02:06:23.585486 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.585537 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.585601 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.585652 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-4003to65535-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.585769 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.585874 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0216 02:06:23.585901 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0216 02:06:23.585992 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.privatekopeio.example.com-ingress-tcp-22to22-bastion.privatekopeio.example.com" }] }] } I0216 02:06:23.586104 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0216 02:06:23.586166 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.586208 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586300 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-4"] }] } I0216 02:06:23.586323 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586402 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586487 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.privatekopeio.example.com" }] }] } I0216 02:06:23.586554 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-nodes.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586630 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:23.586657 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-ingress-tcp-22to22-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586733 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.586835 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-2382to4000-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.586956 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0216 02:06:23.587071 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:23.587105 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.587187 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.587244 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0216 02:06:23.587269 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.privatekopeio.example.com-ingress-all-0to0-masters.privatekopeio.example.com" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" }] }] } I0216 02:06:23.587373 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.587436 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.privatekopeio.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:23.587509 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.privatekopeio.example.com-ingress-tcp-1to2379-masters.privatekopeio.example.com" }] }] } I0216 02:06:23.587591 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:23.587616 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.privatekopeio.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "privatekopeio.example.com" },{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "owned" }] }] } I0216 02:06:23.587685 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:23.587711 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0216 02:06:23.587773 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-3", SubnetId: "subnet-4" } I0216 02:06:23.587805 46260 tags.go:42] CreateTags { Resources: ["nat-b2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0216 02:06:23.587838 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0216 02:06:23.587871 46260 tags.go:42] CreateTags { Resources: ["subnet-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0216 02:06:23.587896 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:23.587924 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-b2345678" }] } I0216 02:06:23.587950 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.588047 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0216 02:06:23.588090 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.588177 46260 tags.go:42] CreateTags { Resources: ["nat-a2345678"], Tags: [{ Key: "kubernetes.io/cluster/privatekopeio.example.com", Value: "shared" }] } I0216 02:06:23.588209 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.588237 46260 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0216 02:06:23.588258 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0216 02:06:23.588283 46260 tags.go:42] CreateTags { Resources: ["rtb-3"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-a2345678" }] } I0216 02:06:23.588308 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:23.588554 46260 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0216 02:06:23.588600 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0216 02:06:23.588626 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.588682 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-a2345678", RouteTableId: "rtb-3" } I0216 02:06:23.588729 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.588724 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.588858 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-b2345678", RouteTableId: "rtb-2" } I0216 02:06:23.588918 46260 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0216 02:06:23.589025 46260 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0216 02:06:23.589060 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.589097 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.589144 46260 dns.go:238] Pre-creating DNS records I0216 02:06:23.589170 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.595068 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:23.597928 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.597996 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.598058 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.671192 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.745323 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.746841 46260 executor.go:111] Tasks: 0 done / 126 total; 47 can run I0216 02:06:23.747222 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.747341 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:23.747406 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.privatekopeio.example.com"] }] } I0216 02:06:23.747541 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.747632 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.privatekopeio.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.747788 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.privatekopeio.example.com"] }] } I0216 02:06:23.747930 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.748025 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:23.748054 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:23.748105 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.749193 46260 executor.go:111] Tasks: 47 done / 126 total; 31 can run I0216 02:06:23.749343 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.privatekopeio.example.com"] } ] } I0216 02:06:23.749499 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0216 02:06:23.749465 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.749557 46260 iamrole.go:161] ListAttachedRolePolicies: masters.privatekopeio.example.com I0216 02:06:23.749599 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0216 02:06:23.749561 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion-elb.privatekopeio.example.com"] } ] } I0216 02:06:23.749763 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:23.749811 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["bastion.privatekopeio.example.com"] } ] } I0216 02:06:23.749976 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.750034 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750165 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750265 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750364 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750470 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-4"] } I0216 02:06:23.750506 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750592 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.750656 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1b.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.750745 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:23.750780 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] },{ Name: "tag:Name", Values: ["utility-us-test-1b.privatekopeio.example.com"] }] } I0216 02:06:23.750875 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:23.750909 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.751645 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.privatekopeio.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.privatekopeio.example.com"] } ] } I0216 02:06:23.751856 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:23.751916 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }, { Name: "tag:Name", Values: ["nodes.privatekopeio.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.privatekopeio.example.com"] } ] } I0216 02:06:23.752027 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:23.752089 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.privatekopeio.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.752294 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:23.755371 46260 executor.go:111] Tasks: 78 done / 126 total; 39 can run W0216 02:06:23.755625 46260 api.go:53] PageSize not implemented I0216 02:06:23.755635 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.755651 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.755752 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.755793 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } W0216 02:06:23.755875 46260 api.go:53] PageSize not implemented I0216 02:06:23.755820 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.755894 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.755967 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e"] } I0216 02:06:23.755999 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0216 02:06:23.756000 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.756184 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.756306 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.756420 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.756555 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0216 02:06:23.756682 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0216 02:06:23.756782 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.756836 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.756914 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.756981 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:23.757024 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.757140 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } I0216 02:06:23.758854 46260 executor.go:111] Tasks: 117 done / 126 total; 6 can run I0216 02:06:23.758927 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0216 02:06:23.758985 46260 api.go:53] PageSize not implemented I0216 02:06:23.758956 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.759036 46260 dnsname.go:121] AliasTarget for "api.privatekopeio.example.com" is "api-privatekopeio-example-tl2bv8.elb.cloudmock.com" I0216 02:06:23.759012 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-3"] } I0216 02:06:23.759011 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } W0216 02:06:23.759169 46260 api.go:53] PageSize not implemented I0216 02:06:23.759186 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.759275 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.759339 46260 executor.go:111] Tasks: 123 done / 126 total; 3 can run I0216 02:06:23.759457 46260 executor.go:111] Tasks: 126 done / 126 total; 0 can run I0216 02:06:23.762003 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:23.762046 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:23.762066 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:23.762084 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:23.762104 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:23.762126 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:23.762144 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:23.762162 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:23.762180 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:23.762201 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:23.762223 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:23.762245 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:23.762273 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0216 02:06:23.762295 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:23.762315 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:23.762333 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:23.762354 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:23.762373 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:23.762393 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-3"] }] } I0216 02:06:23.762414 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:23.762435 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:23.762455 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:23.762475 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:23.762495 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:23.762515 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.762537 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:23.762555 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:23.762576 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0216 02:06:23.762597 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-4"] }] } I0216 02:06:23.762622 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:23.762641 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:23.762659 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:23.762680 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:23.762701 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:23.762721 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:23.763701 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:23.763719 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:23.763747 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.763790 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0216 02:06:23.763838 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.763875 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.763905 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.763945 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.763985 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.764016 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } I0216 02:06:23.764059 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.764108 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:23.764129 46260 natgateway.go:117] DescribeNatGateways: { } I0216 02:06:23.764154 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["privatekopeio.example.com"] }] } I0216 02:06:23.764180 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/privatekopeio.example.com"] }] } W0216 02:06:23.764211 46260 api.go:53] PageSize not implemented I0216 02:06:23.764222 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-privatekopeio-exa-d8ef8e","api-privatekopeio-example-tl2bv8"] } I0216 02:06:23.764255 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:23.764272 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:23.764284 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:23.764298 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:23.764310 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:23.764340 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.764358 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.764387 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:23.764414 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.764444 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:23.764475 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:23.764504 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-3","rtb-1"] } I0216 02:06:23.764529 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-b2345678"] } I0216 02:06:23.764547 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-a2345678"] } autoscaling-group:nodes.privatekopeio.example.com ok autoscaling-group:master-us-test-1a.masters.privatekopeio.example.com ok autoscaling-group:bastion.privatekopeio.example.com ok I0216 02:06:23.764952 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:23.764954 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-privatekopeio-exa-d8ef8e" } I0216 02:06:23.765015 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } load-balancer:bastion-privatekopeio-exa-d8ef8e ok I0216 02:06:23.765036 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } internet-gateway:igw-2 ok I0216 02:06:23.765045 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.765092 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.privatekopeio.example.com" } iam-instance-profile:masters.privatekopeio.example.com ok I0216 02:06:23.765057 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-privatekopeio-example-tl2bv8" } load-balancer:api-privatekopeio-example-tl2bv8 ok I0216 02:06:23.765125 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0216 02:06:23.765015 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-privatekopeio-example-tl2bv8.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.privatekopeio.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.privatekopeio.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.privatekopeio.example.com ok I0216 02:06:23.765125 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } I0216 02:06:23.765186 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.privatekopeio.example.com" } W0216 02:06:23.765219 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions iam-instance-profile:bastions.privatekopeio.example.com ok I0216 02:06:23.765235 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } I0216 02:06:23.765231 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } security-group:sg-2 ok I0216 02:06:23.765261 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.privatekopeio.example.com" } iam-instance-profile:nodes.privatekopeio.example.com ok I0216 02:06:23.765273 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok autoscaling-config:lt-3 ok I0216 02:06:23.765361 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:23.765414 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:23.765455 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.765505 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-4" }] } ] } W0216 02:06:23.765639 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.765658 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0216 02:06:23.765690 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:23.765733 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-4" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0216 02:06:23.765809 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.765822 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0216 02:06:23.765900 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:23.765908 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.privatekopeio.example.com" } I0216 02:06:23.765981 46260 iamrole.go:180] ListAttachedRolePolicies: bastions.privatekopeio.example.com I0216 02:06:23.765953 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0216 02:06:23.766012 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.766022 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:23.765995 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.privatekopeio.example.com", RoleName: "bastions.privatekopeio.example.com" } I0216 02:06:23.766045 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:23.766054 46260 iamrole.go:137] DeleteRole: { RoleName: "bastions.privatekopeio.example.com" } iam-role:bastions.privatekopeio.example.com ok I0216 02:06:23.766081 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-4" } subnet:subnet-4 ok I0216 02:06:23.766095 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.privatekopeio.example.com" } I0216 02:06:23.766114 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0216 02:06:23.766135 46260 iamrole.go:180] ListAttachedRolePolicies: masters.privatekopeio.example.com I0216 02:06:23.766138 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok I0216 02:06:23.766169 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:23.766143 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.privatekopeio.example.com", RoleName: "masters.privatekopeio.example.com" } I0216 02:06:23.766222 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.privatekopeio.example.com" } iam-role:masters.privatekopeio.example.com ok I0216 02:06:23.766204 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 },{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } W0216 02:06:23.766264 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:23.766272 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0216 02:06:23.766259 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.privatekopeio.example.com" } I0216 02:06:23.766305 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.privatekopeio.example.com I0216 02:06:23.766322 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.privatekopeio.example.com", RoleName: "nodes.privatekopeio.example.com" } I0216 02:06:23.766352 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.privatekopeio.example.com" } iam-role:nodes.privatekopeio.example.com ok I0216 02:06:23.766409 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0216 02:06:23.766459 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-3" } route-table:rtb-3 ok I0216 02:06:23.766507 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:23.766572 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:23.766631 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:23.766836 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateKopeio (0.39s) === RUN TestLifecycleIPv6 I0216 02:06:23.767075 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:23.767256 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:23.767318 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:23.767356 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:23.767402 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:23.767439 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:23.767469 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:23.767509 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:23.767534 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:23.767586 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster minimal-ipv6.example.com I0216 02:06:23.782865 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:23.784957 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.785025 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.858695 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.976567 46260 iamrole.go:89] ListRoles: { } I0216 02:06:23.979895 46260 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0216 02:06:23.980189 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:23.980216 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.980378 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } I0216 02:06:23.980367 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.980464 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0216 02:06:23.980609 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:23.980786 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:23.980789 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } I0216 02:06:23.980838 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.980876 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] } I0216 02:06:23.980965 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:23.981018 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.981204 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.981190 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.981277 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:23.981315 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:23.981347 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:23.981389 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.981429 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:23.981589 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.981786 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "minimal-ipv6.example.com" }] }] } I0216 02:06:23.981911 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-events.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:23.981998 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:23.982304 46260 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0216 02:06:23.982374 46260 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0216 02:06:23.982445 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0216 02:06:23.982393 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0216 02:06:23.982513 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0216 02:06:23.982528 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0216 02:06:23.982558 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.982621 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0216 02:06:23.982659 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:23.982651 46260 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-minimal-ipv6-example--bne5ih", Port: 443, Protocol: "TCP", Tags: [{ Key: "Name", Value: "tcp-minimal-ipv6-example--bne5ih" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0216 02:06:23.982728 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:23.982801 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:23.982833 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:23.982787 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "minimal-ipv6.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:23.982857 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0216 02:06:23.982980 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:23.982991 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0216 02:06:23.983034 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:23.983083 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:23.983108 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.983171 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public-us-test-1a"] }] } I0216 02:06:23.983246 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" } ] }], VpcId: "vpc-1" } I0216 02:06:23.983245 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0216 02:06:23.983443 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.983540 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:23.983576 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.983753 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:23.983876 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:23.983909 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:23.983937 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public-us-test-1a" }, { Key: "Name", Value: "public-us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" } ] } I0216 02:06:23.984007 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.minimal-ipv6.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:23.984059 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:23.984089 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:23.984112 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:23.984139 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0216 02:06:23.984179 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.984224 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:23.984079 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/minimal-ipv6.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/minimal-ipv6.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AssignIpv6Addresses\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeNetworkInterfaces\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"minimal-ipv6.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0216 02:06:23.985442 46260 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0216 02:06:23.985533 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.985609 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.985691 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:23.985768 46260 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.985829 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", Ipv6CidrBlock: "2001:db8:0:111::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" }, { Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }, { Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" } ] }], VpcId: "vpc-1" } I0216 02:06:23.985917 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.985947 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:23.985983 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }] } I0216 02:06:23.986101 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0216 02:06:23.986234 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.986424 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" }] }] } I0216 02:06:23.986567 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.986670 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.986721 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.986766 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:23.986811 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0216 02:06:23.986996 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0216 02:06:23.987078 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:23.987133 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0216 02:06:23.987205 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" }] }] } I0216 02:06:23.987298 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-nodes.minimal-ipv6.example.com" }] }] } I0216 02:06:23.987382 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-::/0-ingress-tcp-22to22-nodes.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0216 02:06:23.987458 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0216 02:06:23.987488 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-2" } I0216 02:06:23.987515 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-1to2379-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.987594 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-4003to65535-masters.minimal-ipv6.example.com" }] }] } I0216 02:06:23.987671 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.987815 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.987876 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.987920 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-tcp-2382to4000-masters.minimal-ipv6.example.com" }] }] } I0216 02:06:23.988009 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-::/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.988089 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.minimal-ipv6.example.com-ingress-udp-1to65535-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.988170 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.988245 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:23.988271 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" }] }] } I0216 02:06:23.988354 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.minimal-ipv6.example.com-ingress-all-0to0-masters.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.988427 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.988540 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.990262 46260 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0216 02:06:23.990388 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.990437 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:23.990529 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:23.990599 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.990704 46260 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.990776 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:23.990748 46260 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "dualstack", Name: "api-minimal-ipv6-example--jhj9te", SubnetMappings: [{ SubnetId: "subnet-1" }], Tags: [{ Key: "Name", Value: "api.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }], Type: "network" } I0216 02:06:23.990824 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-1" } I0216 02:06:23.990814 46260 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-1", SubnetId: "subnet-1", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.minimal-ipv6.example.com" },{ Key: "KubernetesCluster", Value: "minimal-ipv6.example.com" },{ Key: "kubernetes.io/cluster/minimal-ipv6.example.com", Value: "owned" }] }] } I0216 02:06:23.990842 46260 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1", Port: 443, Protocol: "TCP" } I0216 02:06:23.990875 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:23.990900 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:23.990896 46260 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "false" },{ Key: "access_logs.s3.enabled", Value: "false" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0216 02:06:23.990928 46260 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0216 02:06:23.990975 46260 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0216 02:06:23.991024 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.991074 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:23.991119 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.991184 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.991206 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "64:ff9b::/96", NatGatewayId: "nat-1", RouteTableId: "rtb-2" } I0216 02:06:23.991260 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:23.991321 46260 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0216 02:06:23.991356 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:23.991376 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.991408 46260 dns.go:238] Pre-creating DNS records I0216 02:06:23.991423 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:23.995804 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:23.997810 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:23.997875 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.072264 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.193091 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.194408 46260 executor.go:111] Tasks: 0 done / 106 total; 45 can run I0216 02:06:24.194554 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.194874 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.194993 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.195085 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:24.195060 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.195246 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.minimal-ipv6.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.195370 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.195474 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:24.195503 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:24.195550 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.196768 46260 executor.go:111] Tasks: 45 done / 106 total; 21 can run I0216 02:06:24.196867 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["public-us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.196949 46260 iamrole.go:161] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0216 02:06:24.196927 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-minimal-ipv6-example--bne5ih"] } I0216 02:06:24.196989 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0216 02:06:24.196980 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0216 02:06:24.196990 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.minimal-ipv6.example.com"] } ] } I0216 02:06:24.197112 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.197146 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.197202 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "tag:Name", Values: ["nodes.minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.minimal-ipv6.example.com"] } ] } I0216 02:06:24.197283 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.197322 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.197428 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.197559 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.minimal-ipv6.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.minimal-ipv6.example.com"] } ] } I0216 02:06:24.197737 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:24.197839 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.199770 46260 executor.go:111] Tasks: 66 done / 106 total; 30 can run I0216 02:06:24.199979 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.200065 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.200169 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.200263 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.200423 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.minimal-ipv6.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.200575 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:24.200633 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.200712 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:24.200747 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } I0216 02:06:24.200782 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.200865 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.200951 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.201058 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.201095 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0216 02:06:24.203186 46260 executor.go:111] Tasks: 96 done / 106 total; 5 can run I0216 02:06:24.203251 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.203365 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.203374 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:24.203460 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:24.203468 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:24.203483 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0216 02:06:24.203549 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0216 02:06:24.203611 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0216 02:06:24.203659 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4"] } I0216 02:06:24.203695 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } I0216 02:06:24.203839 46260 executor.go:111] Tasks: 101 done / 106 total; 5 can run I0216 02:06:24.203941 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:24.204036 46260 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0216 02:06:24.204060 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:24.204094 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0216 02:06:24.204139 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.204254 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:24.204445 46260 dnsname.go:121] AliasTarget for "api.minimal-ipv6.example.com" is "api-minimal-ipv6-example--jhj9te.amazonaws.com" I0216 02:06:24.204626 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:24.204871 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0216 02:06:24.204978 46260 executor.go:111] Tasks: 106 done / 106 total; 0 can run I0216 02:06:24.208195 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:24.208252 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:24.208287 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:24.208320 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:24.208354 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:24.208391 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0216 02:06:24.208426 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:24.208456 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:24.208486 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:24.208520 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:24.208555 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:24.208590 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:24.208630 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:24.208671 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:24.208705 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:24.208738 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:24.208772 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:24.208808 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:24.208840 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:24.208874 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:24.208909 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:24.208943 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:24.208976 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:24.209018 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:24.209050 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:24.209082 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:24.209114 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:24.209150 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:24.209187 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:24.209222 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:24.211086 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:24.211114 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:24.211154 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211209 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0216 02:06:24.211268 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211321 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211368 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211416 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211452 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211495 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } I0216 02:06:24.211543 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211588 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:24.211615 46260 natgateway.go:117] DescribeNatGateways: { } I0216 02:06:24.211653 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["minimal-ipv6.example.com"] }] } I0216 02:06:24.211691 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/minimal-ipv6.example.com"] }] } W0216 02:06:24.211739 46260 api.go:53] PageSize not implemented I0216 02:06:24.211757 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:24.211780 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:24.211793 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1"] } I0216 02:06:24.211827 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:24.211849 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:24.211865 46260 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" ] } I0216 02:06:24.211916 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.211942 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.211983 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:24.212015 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.212052 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:24.212096 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:24.212135 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0216 02:06:24.212169 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:24.212199 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-1"] } autoscaling-group:master-us-test-1a.masters.minimal-ipv6.example.com ok autoscaling-group:nodes.minimal-ipv6.example.com ok I0216 02:06:24.212717 46260 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } I0216 02:06:24.212788 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:24.212817 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0216 02:06:24.212888 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.minimal-ipv6.example.com" } nat-gateway:nat-1 ok I0216 02:06:24.212879 46260 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-minimal-ipv6-example--jhj9te/1 ok iam-instance-profile:nodes.minimal-ipv6.example.com ok I0216 02:06:24.212843 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.212809 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [ { Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { Name: "api.internal.minimal-ipv6.example.com.", ResourceRecords: [{ Value: "fd00:dead:add::" }], TTL: 10, Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "AAAA" } }, { Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-minimal-ipv6-example--jhj9te.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.minimal-ipv6.example.com", Type: "A" } } ] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/AAAA/kops-controller.internal.minimal-ipv6.example.com. ok I0216 02:06:24.212932 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0216 02:06:24.212957 46260 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-minimal-ipv6-example--bne5ih/4 ok I0216 02:06:24.213007 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.minimal-ipv6.example.com" } iam-instance-profile:masters.minimal-ipv6.example.com ok I0216 02:06:24.212981 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 } ] } W0216 02:06:24.213178 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:24.213209 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:24.213268 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok autoscaling-config:lt-2 ok I0216 02:06:24.213373 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:24.213443 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0216 02:06:24.213478 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:24.213529 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: -1, IpProtocol: "icmpv6", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: -1 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }, { FromPort: 22, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", Ipv6Ranges: [{ CidrIpv6: "::/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] } ] } W0216 02:06:24.213787 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:24.213808 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0216 02:06:24.213843 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.213891 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0216 02:06:24.213942 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0216 02:06:24.214012 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:24.214094 46260 address.go:185] ReleaseAddress: { AllocationId: "eipalloc-1" } I0216 02:06:24.214127 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.minimal-ipv6.example.com" } I0216 02:06:24.214153 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } elastic-ip:eipalloc-1 ok subnet:subnet-1 ok I0216 02:06:24.214182 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.minimal-ipv6.example.com I0216 02:06:24.214233 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.minimal-ipv6.example.com", RoleName: "nodes.minimal-ipv6.example.com" } I0216 02:06:24.214280 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.minimal-ipv6.example.com" } iam-role:nodes.minimal-ipv6.example.com ok I0216 02:06:24.214327 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.minimal-ipv6.example.com" } I0216 02:06:24.214368 46260 iamrole.go:180] ListAttachedRolePolicies: masters.minimal-ipv6.example.com I0216 02:06:24.214378 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.minimal-ipv6.example.com", RoleName: "masters.minimal-ipv6.example.com" } I0216 02:06:24.214398 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.minimal-ipv6.example.com" } iam-role:masters.minimal-ipv6.example.com ok I0216 02:06:24.214447 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0216 02:06:24.214494 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:24.214542 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:24.214775 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleIPv6 (0.45s) === RUN TestLifecycleSharedVPC I0216 02:06:24.215009 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:24.215174 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:24.215203 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:24.215228 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:24.215259 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:24.215285 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:24.215303 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:24.215328 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:24.215344 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:24.215378 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedvpc.example.com I0216 02:06:24.231781 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:24.233521 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.233576 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.307345 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.372994 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.374058 46260 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0216 02:06:24.374216 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.374282 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } I0216 02:06:24.375219 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:24.376238 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:24.374377 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0216 02:06:24.376365 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:24.376466 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:24.376518 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:24.376705 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } I0216 02:06:24.376840 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.376900 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] } I0216 02:06:24.376984 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.376999 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:24.377182 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.377353 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.378636 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:24.378730 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:24.379366 46260 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0216 02:06:24.379448 46260 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0216 02:06:24.379509 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0216 02:06:24.379481 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0216 02:06:24.379600 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0216 02:06:24.379684 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0216 02:06:24.379674 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0216 02:06:24.379840 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0216 02:06:24.379877 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.379971 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:24.380037 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.380083 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-12345678" } I0216 02:06:24.380165 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:24.380174 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-12345678" } I0216 02:06:24.380204 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "sharedvpc.example.com" }, { Key: "KubernetesCluster", Value: "sharedvpc.example.com" }, { Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0216 02:06:24.380277 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:24.380373 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:24.380462 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:24.380505 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.380543 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedvpc.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:24.380644 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:24.380723 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.380773 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:24.380815 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:24.380825 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedvpc.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedvpc.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedvpc.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0216 02:06:24.382270 46260 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0216 02:06:24.382395 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0216 02:06:24.382554 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.382698 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.382767 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.382834 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.382953 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:24.382982 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:24.383007 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-1to2379-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.383154 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-4003to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.383242 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.383328 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0216 02:06:24.383445 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-udp-1to65535-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0216 02:06:24.383619 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.383876 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.384037 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.384110 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.384555 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedvpc.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" }] }] } I0216 02:06:24.384818 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385000 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.385119 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385302 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedvpc.example.com-ingress-tcp-2382to4000-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385453 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385582 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385718 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedvpc.example.com-ingress-all-0to0-masters.sharedvpc.example.com" },{ Key: "KubernetesCluster", Value: "sharedvpc.example.com" },{ Key: "kubernetes.io/cluster/sharedvpc.example.com", Value: "owned" }] }] } I0216 02:06:24.385852 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-1" } I0216 02:06:24.385911 46260 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0216 02:06:24.386130 46260 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0216 02:06:24.386250 46260 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0216 02:06:24.386290 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.386331 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.386387 46260 dns.go:238] Pre-creating DNS records I0216 02:06:24.386415 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.392461 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:24.395327 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.395411 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.468838 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.535790 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.536809 46260 executor.go:111] Tasks: 0 done / 84 total; 42 can run I0216 02:06:24.537344 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.537412 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:24.537559 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:24.537643 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:24.537676 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:24.538026 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.538112 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedvpc.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:24.538175 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.539677 46260 executor.go:111] Tasks: 42 done / 84 total; 17 can run I0216 02:06:24.539799 46260 iamrole.go:161] ListAttachedRolePolicies: masters.sharedvpc.example.com I0216 02:06:24.539842 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0216 02:06:24.539789 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedvpc.example.com"] } ] } I0216 02:06:24.539983 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.540043 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:24.540152 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.540275 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedvpc.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedvpc.example.com"] } ] } I0216 02:06:24.540421 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.540474 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.sharedvpc.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.540612 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:24.544219 46260 executor.go:111] Tasks: 59 done / 84 total; 21 can run I0216 02:06:24.544393 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.544480 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.544585 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.544731 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.544866 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.544983 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.545199 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.549104 46260 executor.go:111] Tasks: 80 done / 84 total; 2 can run I0216 02:06:24.549287 46260 executor.go:111] Tasks: 82 done / 84 total; 2 can run I0216 02:06:24.549373 46260 executor.go:111] Tasks: 84 done / 84 total; 0 can run I0216 02:06:24.552427 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:24.552490 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:24.552521 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:24.552555 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:24.552586 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:24.552618 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:24.552650 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:24.552679 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:24.552716 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:24.552757 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:24.552793 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:24.552824 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:24.552855 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:24.552891 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:24.552923 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:24.552956 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:24.552992 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:24.553028 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:24.553059 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:24.553090 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:24.553120 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:24.553155 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:24.553189 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:24.554723 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:24.554747 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:24.554783 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.554831 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0216 02:06:24.554885 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.554941 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.554980 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.555017 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.555054 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.555090 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } I0216 02:06:24.555134 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.555186 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedvpc.example.com"] }] } I0216 02:06:24.555221 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedvpc.example.com"] }] } W0216 02:06:24.555265 46260 api.go:53] PageSize not implemented I0216 02:06:24.555287 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:24.555311 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:24.555327 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:24.555348 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:24.555365 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:24.555409 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.555435 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.555473 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:24.555510 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.555550 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:24.555582 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:24.555608 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.555968 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0216 02:06:24.556030 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedvpc.example.com" } autoscaling-group:master-us-test-1a.masters.sharedvpc.example.com ok iam-instance-profile:nodes.sharedvpc.example.com ok I0216 02:06:24.556022 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.556060 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0216 02:06:24.556106 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedvpc.example.com" } autoscaling-group:nodes.sharedvpc.example.com ok iam-instance-profile:masters.sharedvpc.example.com ok I0216 02:06:24.556049 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedvpc.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.sharedvpc.example.com. ok I0216 02:06:24.556099 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0216 02:06:24.556250 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:24.556273 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-2 ok I0216 02:06:24.556327 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:24.556385 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:24.556430 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.556472 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0216 02:06:24.556686 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:24.556709 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0216 02:06:24.556762 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0216 02:06:24.556855 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedvpc.example.com" } I0216 02:06:24.556900 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedvpc.example.com I0216 02:06:24.556882 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:24.556914 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedvpc.example.com", RoleName: "nodes.sharedvpc.example.com" } I0216 02:06:24.556955 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedvpc.example.com" } iam-role:nodes.sharedvpc.example.com ok I0216 02:06:24.556991 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedvpc.example.com" } I0216 02:06:24.557045 46260 iamrole.go:180] ListAttachedRolePolicies: masters.sharedvpc.example.com I0216 02:06:24.557055 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedvpc.example.com", RoleName: "masters.sharedvpc.example.com" } I0216 02:06:24.557076 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedvpc.example.com" } iam-role:masters.sharedvpc.example.com ok I0216 02:06:24.557126 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:24.557342 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedVPC (0.34s) === RUN TestLifecycleComplex I0216 02:06:24.557569 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:24.557768 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:24.557801 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:24.557826 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:24.557857 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:24.557884 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:24.557901 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:24.557926 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:24.557944 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:24.557979 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster complex.example.com I0216 02:06:24.579178 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:24.581612 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.581676 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.655205 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.777868 46260 iamrole.go:89] ListRoles: { } I0216 02:06:24.779297 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0216 02:06:24.779529 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:24.779586 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0216 02:06:24.779797 46260 api.go:53] PageSize not implemented I0216 02:06:24.779756 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } I0216 02:06:24.779778 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.779867 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.complex.example.com", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] } I0216 02:06:24.779921 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.779970 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } I0216 02:06:24.780033 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", Tags: [ { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0216 02:06:24.780116 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-main.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/main", Value: "a/a" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:24.780252 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:24.780301 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.780460 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:24.780454 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0216 02:06:24.780555 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:24.780584 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:24.780612 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:24.780673 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.780703 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "k8s.io/etcd/events", Value: "a/a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Name", Value: "a.etcd-events.complex.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:24.780821 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:24.780861 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:24.780960 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:24.781050 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.781154 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:24.781193 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:24.781529 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:06:24.781632 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:24.781709 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:24.781703 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0216 02:06:24.781716 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0216 02:06:24.781797 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:24.781840 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:24.781801 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.781862 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:24.781889 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0216 02:06:24.781923 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.781901 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-test-1a.complex.example.com" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0216 02:06:24.782069 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:24.782034 46260 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tls-complex-example-com-5nursn", Port: 443, Protocol: "TLS", Tags: [ { Key: "Name", Value: "tls-complex-example-com-5nursn" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0216 02:06:24.782114 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:24.782161 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0216 02:06:24.782171 46260 targetgroups.go:94] CreateTargetGroup { HealthyThresholdCount: 2, Name: "tcp-complex-example-com-vpjolq", Port: 443, Protocol: "TCP", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "tcp-complex-example-com-vpjolq" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ], UnhealthyThresholdCount: 2, VpcId: "vpc-1" } I0216 02:06:24.782313 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api-elb.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0216 02:06:24.782463 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:24.782518 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:24.782570 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.782676 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:24.782761 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.782804 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0216 02:06:24.782902 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:24.782947 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.782931 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.64.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-private.complex.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:24.783024 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.783118 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.783211 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:24.783324 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:24.783397 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:24.783485 46260 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.1.0.0/16", VpcId: "vpc-1" } I0216 02:06:24.783529 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:24.783572 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:24.783614 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }], VpcId: "vpc-1" } I0216 02:06:24.783590 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.96.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "Name", Value: "us-east-1a-utility.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }], VpcId: "vpc-1" } I0216 02:06:24.783709 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] } I0216 02:06:24.783829 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:24.783577 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/complex.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/complex.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"complex.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0216 02:06:24.783877 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:24.783951 46260 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "10.2.0.0/16", VpcId: "vpc-1" } I0216 02:06:24.784018 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0216 02:06:24.784162 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [ { Key: "Name", Value: "complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.784166 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }], VpcId: "vpc-1" } I0216 02:06:24.784263 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:24.784315 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" }, { Key: "Name", Value: "private-us-test-1a.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] } I0216 02:06:24.784419 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:24.784474 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:24.784528 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:24.784583 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0216 02:06:24.784635 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:24.784674 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:24.784714 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.complex.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }], VpcId: "vpc-1" } I0216 02:06:24.784814 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:24.784861 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:24.785717 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:06:24.785830 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.785894 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:24.785969 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:24.785927 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.786028 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.786124 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.786100 46260 loadbalancers.go:80] CreateLoadBalancer v2 { IpAddressType: "ipv4", Name: "api-complex-example-com-vd3t5n", SubnetMappings: [{ AllocationId: "eipalloc-012345a678b9cdefa", SubnetId: "subnet-1" }], Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "api.complex.example.com" } ], Type: "network" } I0216 02:06:24.786226 46260 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-test-1:000000000000:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0216 02:06:24.786316 46260 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0216 02:06:24.786206 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0216 02:06:24.786380 46260 loadbalancers.go:158] ModifyLoadBalancerAttributes v2 { Attributes: [{ Key: "load_balancing.cross_zone.enabled", Value: "true" },{ Key: "access_logs.s3.enabled", Value: "true" },{ Key: "access_logs.s3.bucket", Value: "access-log-example" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:24.786479 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.786643 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", RouteTableId: "rtb-2", TransitGatewayId: "tgw-123456" } I0216 02:06:24.786788 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }] } I0216 02:06:24.786981 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.complex.example.com" } ] }] } I0216 02:06:24.787132 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.787228 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "192.168.1.10/32", RouteTableId: "rtb-2", TransitGatewayId: "tgw-0123456" } I0216 02:06:24.787272 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:24.787362 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0216 02:06:24.787467 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-2382to4000-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0216 02:06:24.787630 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }] } I0216 02:06:24.788024 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-3"] }] } I0216 02:06:24.788069 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }] } I0216 02:06:24.788169 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:24.788240 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0216 02:06:24.788399 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-ingress-udp-1to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0216 02:06:24.788567 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }] } I0216 02:06:24.788663 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-0.0.0.0/0" } ] }] } I0216 02:06:24.788806 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:24.788861 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.789007 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:24.789049 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.789202 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" } ] }] } I0216 02:06:24.789356 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }] } I0216 02:06:24.789447 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-4003to65535-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.789595 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Name", Value: "from-nodes.complex.example.com-ingress-tcp-1to2379-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" } ] }] } I0216 02:06:24.789740 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Owner", Value: "John Doe" }, { Key: "Name", Value: "from-1.1.1.1/32-ingress-tcp-22to22-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0216 02:06:24.789881 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-1.1.1.0/24-ingress-tcp-443to443-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0216 02:06:24.790041 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.790138 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.790213 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0216 02:06:24.790258 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-egress-all-0to0-::/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" } ] }] } I0216 02:06:24.790405 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0216 02:06:24.790558 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-nodes.complex.example.com-egress-all-0to0-0.0.0.0/0" }, { Key: "KubernetesCluster", Value: "complex.example.com" }, { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" } ] }] } I0216 02:06:24.790716 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-3" } I0216 02:06:24.790760 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [ { Key: "kubernetes.io/cluster/complex.example.com", Value: "owned" }, { Key: "Owner", Value: "John Doe" }, { Key: "foo/bar", Value: "fib+baz" }, { Key: "Name", Value: "from-masters.complex.example.com-ingress-all-0to0-nodes.complex.example.com" }, { Key: "KubernetesCluster", Value: "complex.example.com" } ] }] } I0216 02:06:24.790902 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }] } I0216 02:06:24.790999 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }] } I0216 02:06:24.791099 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:24.791152 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 }] } I0216 02:06:24.791248 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:24.791288 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.791343 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.791393 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0216 02:06:24.799444 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:06:24.799566 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:24.799798 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:24.799911 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:06:24.800037 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0216 02:06:24.800079 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:24.800113 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.800163 46260 dns.go:238] Pre-creating DNS records I0216 02:06:24.800186 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:24.808042 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:24.810842 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.810914 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:24.884820 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.027656 46260 iamrole.go:89] ListRoles: { } I0216 02:06:25.030006 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0216 02:06:25.030285 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0216 02:06:25.030518 46260 api.go:53] PageSize not implemented I0216 02:06:25.030504 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:25.030638 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:25.030717 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.030768 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:25.030886 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:25.030971 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:25.030363 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:25.031070 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:25.031257 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.032324 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.032732 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.035155 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:06:25.035305 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:25.035259 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:25.035414 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.035508 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:25.035501 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:25.035790 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:25.035880 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.035550 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.035953 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:25.035993 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.036025 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.036077 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.036210 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:25.036251 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0216 02:06:25.036367 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.036441 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.036546 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.036655 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:25.036721 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.036828 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:25.036922 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:25.036951 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.037034 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:25.037058 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:25.037156 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:25.038967 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:06:25.039087 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.039182 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:25.039265 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:25.039238 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.039283 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.039327 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.039362 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.039346 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.039396 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.039430 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.039452 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.039462 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.039563 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.039653 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.039903 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.040032 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.040222 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.040348 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.040451 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.042169 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:06:25.042275 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:25.042354 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:25.042376 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:25.042399 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.042475 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:06:25.042604 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.sslCertificate=arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678] I0216 02:06:25.048109 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.050991 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.062414 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:25.066119 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.066207 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.138233 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.257431 46260 iamrole.go:89] ListRoles: { } I0216 02:06:25.258818 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0216 02:06:25.259105 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:25.259189 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } W0216 02:06:25.259161 46260 api.go:53] PageSize not implemented I0216 02:06:25.259307 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:25.259380 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:25.259420 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:25.259489 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.259602 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.259681 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:25.259702 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:25.259745 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.259783 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.259861 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:25.260855 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:06:25.260955 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.261040 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:25.260950 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:25.261081 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.261063 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.261127 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:25.261183 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:25.261184 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.261137 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:25.261281 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:25.261346 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.261475 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.261608 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:25.261655 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.261743 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:25.261879 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:25.261923 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.262047 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:25.262082 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.262148 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.262229 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.262304 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:25.262459 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:25.262526 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] }] } I0216 02:06:25.262653 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:25.264649 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:06:25.264765 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.264801 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0216 02:06:25.264824 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } W0216 02:06:25.264868 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:25.264884 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.264921 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.264943 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.264967 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.264996 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.265014 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.264967 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.265114 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.265138 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { Names: ["api-complex-example-com-vd3t5n"] } I0216 02:06:25.265159 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.265175 46260 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/1" } I0216 02:06:25.265194 46260 listeners.go:103] DeleteListener v2 { ListenerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:listener/net/api-complex-example-com-vd3t5n/1/2" } I0216 02:06:25.265213 46260 listeners.go:56] CreateListener v2 { Certificates: [{ CertificateArn: "arn:aws-test:acm:us-east-1:123456789012:certificate/123456789012-1234-1234-1234-12345678" }], DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 443, Protocol: "TLS", SslPolicy: "ELBSecurityPolicy-2016-08" } I0216 02:06:25.265265 46260 listeners.go:56] CreateListener v2 { DefaultActions: [{ TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", Type: "forward" }], LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1", Port: 8443, Protocol: "TCP" } I0216 02:06:25.265302 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.265323 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.265336 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.265415 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.265525 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.265778 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.265861 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.265909 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.266005 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.267627 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:06:25.267749 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:25.267847 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:25.267876 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:25.267901 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.268006 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:06:25.268172 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run I0216 02:06:25.268226 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.268272 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:25.268327 46260 dns.go:238] Pre-creating DNS records I0216 02:06:25.274572 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:25.277061 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.277135 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.351315 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.488501 46260 iamrole.go:89] ListRoles: { } I0216 02:06:25.490754 46260 executor.go:111] Tasks: 0 done / 122 total; 51 can run I0216 02:06:25.490951 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.491083 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:25.491115 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:25.491145 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.491232 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:25.491252 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } W0216 02:06:25.491341 46260 api.go:53] PageSize not implemented I0216 02:06:25.491336 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0216 02:06:25.492030 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.492083 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:25.492172 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.492273 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:25.492347 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:25.496041 46260 executor.go:111] Tasks: 51 done / 122 total; 27 can run I0216 02:06:25.496123 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.496230 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:25.496229 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.496328 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:25.496359 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:25.496298 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.496364 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.496458 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:25.496528 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.496534 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.496690 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:25.496732 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:25.496896 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:25.496952 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.497040 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.497179 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:25.497218 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.497314 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.497443 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:25.497490 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.497572 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:25.497738 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:25.497799 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:25.497926 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:25.497962 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.499531 46260 executor.go:111] Tasks: 78 done / 122 total; 39 can run I0216 02:06:25.499669 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.499787 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:25.499848 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:25.499860 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.499891 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.499914 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.499893 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.499938 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.499959 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.499979 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.499967 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.500055 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.500197 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.500361 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.500447 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.500553 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.500680 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.500821 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.500981 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.502653 46260 executor.go:111] Tasks: 117 done / 122 total; 3 can run I0216 02:06:25.502713 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:25.502766 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:25.502782 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:25.502795 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.502841 46260 executor.go:111] Tasks: 120 done / 122 total; 2 can run I0216 02:06:25.502934 46260 executor.go:111] Tasks: 122 done / 122 total; 0 can run lifecycle_integration_test.go:223: overriding cluster values [spec.api.loadBalancer.additionalSecurityGroups=sg-123456] I0216 02:06:25.505129 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.507229 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.514232 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:25.516647 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.516716 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.589490 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.708974 46260 iamrole.go:89] ListRoles: { } I0216 02:06:25.710454 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0216 02:06:25.710710 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.710772 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0216 02:06:25.710785 46260 api.go:53] PageSize not implemented I0216 02:06:25.710834 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:25.710994 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.711092 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:25.711116 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:25.711150 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.711176 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.711253 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:25.711316 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.711402 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:25.711460 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:25.711512 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0216 02:06:25.712732 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:25.712837 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:25.712827 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.712878 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:25.712931 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.712925 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.713016 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:25.713026 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:25.713065 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.713038 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.713148 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:25.713179 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.713260 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:25.713286 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:25.713380 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:25.713421 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0216 02:06:25.713504 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.713560 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:25.713650 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:25.713733 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.713840 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.713899 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.713980 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:25.714041 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:25.714193 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:25.714240 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.715906 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:25.715992 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.716029 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:25.716074 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:25.716085 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.716108 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.716104 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.716129 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.716193 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.716178 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.716220 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.716242 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.716244 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.716295 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.716384 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.716549 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.716675 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.716816 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.717165 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.717264 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.718258 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.718301 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.719058 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:25.719154 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:25.719240 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:25.719268 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:25.719316 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.719417 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:25.719631 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:25.719717 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.719795 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:25.719883 46260 dns.go:238] Pre-creating DNS records I0216 02:06:25.725483 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:25.727921 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.727986 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.800410 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.946273 46260 iamrole.go:89] ListRoles: { } I0216 02:06:25.948666 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0216 02:06:25.948974 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:25.948933 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.949111 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:25.949192 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } W0216 02:06:25.949280 46260 api.go:53] PageSize not implemented I0216 02:06:25.949300 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:25.949402 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.949502 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] }] } I0216 02:06:25.949580 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:25.949639 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.950155 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:25.950198 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:25.950568 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:25.950706 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.953863 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:25.953943 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:25.953979 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:25.953990 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:25.954049 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.954091 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:25.954096 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:25.954136 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.954095 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.954200 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:25.954381 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:25.954445 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.954573 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:25.954604 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.954692 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.954774 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:25.954798 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.954869 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.954921 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:25.954972 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.955050 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:25.955136 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:25.955160 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.955210 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:25.955297 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:25.955326 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:25.956939 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:25.957021 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.957111 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.957142 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } I0216 02:06:25.957164 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0216 02:06:25.957212 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:25.957230 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.957232 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.957272 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.957313 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.957357 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:25.957403 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:25.957401 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.957445 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:25.957547 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.958143 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.958269 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:25.958415 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:25.958590 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.958680 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.959746 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:25.959832 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:25.959891 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:25.959907 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:25.959922 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:25.959974 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:25.960060 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t3.large] I0216 02:06:25.965112 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:25.968252 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.968351 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.977914 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:25.981650 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.981739 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:25.981804 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.054650 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.175562 46260 iamrole.go:89] ListRoles: { } I0216 02:06:26.177009 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0216 02:06:26.177218 46260 api.go:53] PageSize not implemented I0216 02:06:26.177233 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:26.177367 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.177447 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:26.177469 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:26.177505 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.177532 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.177652 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:26.177796 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.177937 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:26.178034 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:26.178129 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:26.178225 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.178364 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0216 02:06:26.179038 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:26.179120 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.179192 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.179291 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:26.179284 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:26.179336 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:26.179413 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.179437 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:26.179370 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:26.179481 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:26.179489 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:26.179546 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.179614 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:26.179673 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.179765 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.179903 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:26.179941 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:26.180074 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:26.180116 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.180238 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.180319 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.180397 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.180474 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:26.180517 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.180648 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.180743 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:26.182591 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:26.182665 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.182784 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.182835 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:26.182890 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:26.182901 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.182932 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.182954 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.182832 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.182975 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.183093 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.183148 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.183088 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.183298 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.183434 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.183597 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.183652 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.183759 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.183820 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.183906 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.184934 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:26.185053 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:26.185115 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:26.185130 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:26.185145 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.185198 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:26.185282 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:26.185312 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.185333 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:26.185367 46260 dns.go:238] Pre-creating DNS records I0216 02:06:26.189787 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:26.194020 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.194083 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.194119 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.270499 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.390755 46260 iamrole.go:89] ListRoles: { } I0216 02:06:26.392191 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0216 02:06:26.392351 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0216 02:06:26.392409 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:26.392471 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } W0216 02:06:26.392566 46260 api.go:53] PageSize not implemented I0216 02:06:26.392575 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.392690 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:26.392790 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.393578 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:26.393624 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:26.393716 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.393766 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:26.393918 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.394111 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.394308 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:26.397474 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:26.397563 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.397642 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.397755 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:26.397751 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0216 02:06:26.397820 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:26.397838 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.397837 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:26.397896 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.397930 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.397981 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:26.398009 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.398010 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:26.398065 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.398059 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.398141 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:26.398166 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:26.398267 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:26.398306 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:26.398390 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:26.398418 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.398490 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.398536 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:26.398620 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:26.398665 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.398740 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:26.400774 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:26.400889 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.400979 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.401060 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.401044 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:26.401133 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:26.401155 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.401200 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.401238 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.401281 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.401321 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.401301 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.401356 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.401565 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.401673 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.401767 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.401886 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.402074 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.402225 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.402346 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.403978 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:26.404033 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:26.404085 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:26.404102 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:26.404118 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.404197 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:26.404320 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run lifecycle_integration_test.go:251: overriding instance group values (nodes) [spec.mixedInstancesPolicy.instances=t2.medium] I0216 02:06:26.406283 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.408611 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.408665 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.408749 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.414865 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:26.417251 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.417323 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.417361 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.417390 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.491192 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.645591 46260 iamrole.go:89] ListRoles: { } I0216 02:06:26.646968 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run I0216 02:06:26.647115 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.647238 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:26.647275 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:26.647232 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:26.647351 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } W0216 02:06:26.647458 46260 api.go:53] PageSize not implemented I0216 02:06:26.647548 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.647613 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:26.647720 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0216 02:06:26.647828 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:26.647933 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.648094 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.648263 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["complex.example.com"] }] } I0216 02:06:26.648397 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:26.649453 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:26.649550 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:26.649651 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:26.649654 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:26.649713 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:26.649729 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.649679 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.649793 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:26.649834 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.649813 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.649973 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.650128 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:26.650175 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] }] } I0216 02:06:26.650319 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.650463 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.650617 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:26.650683 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.650786 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.650862 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:26.650995 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.651077 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.651154 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:26.651197 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:26.651313 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:26.651361 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:26.651459 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:26.652758 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:26.652846 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.652974 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.653060 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:26.653115 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:26.653072 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.653134 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.653165 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.653187 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.653209 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.653174 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.653231 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.653251 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.653730 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.653845 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.654014 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.654230 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.654327 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.654459 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.654648 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.656807 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:26.656898 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:26.656978 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:26.657005 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:26.657032 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.657119 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:26.657240 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:26.657279 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.657326 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:26.657382 46260 dns.go:238] Pre-creating DNS records I0216 02:06:26.666199 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.24.4 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.24.4 ********************************************************************************* I0216 02:06:26.670430 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.670528 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.670592 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.670673 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.744901 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.964723 46260 iamrole.go:89] ListRoles: { } I0216 02:06:26.966113 46260 executor.go:111] Tasks: 0 done / 123 total; 52 can run W0216 02:06:26.966556 46260 api.go:53] PageSize not implemented I0216 02:06:26.966439 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.966588 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:26.966700 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-main.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.966847 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["a.etcd-events.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.966974 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid3"] } I0216 02:06:26.967098 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid6"] } I0216 02:06:26.967192 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid5"] } I0216 02:06:26.967281 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-exampleid4"] } I0216 02:06:26.967388 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-123456"] } I0216 02:06:26.967477 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.967567 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:26.967595 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:26.967648 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.968335 46260 executor.go:111] Tasks: 52 done / 123 total; 27 can run I0216 02:06:26.968446 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:26.968458 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tcp-complex-example-com-vpjolq"] } I0216 02:06:26.968460 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["api-elb.complex.example.com"] } ] } I0216 02:06:26.968533 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.968597 46260 iamrole.go:161] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:26.968589 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] },{ Name: "tag:Name", Values: ["us-east-1a-utility.complex.example.com"] }] } I0216 02:06:26.968646 46260 targetgroups.go:32] DescribeTargetGroups { Names: ["tls-complex-example-com-5nursn"] } I0216 02:06:26.968715 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.968765 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-3"] } I0216 02:06:26.968821 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.968963 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.969096 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.969188 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-east-1a-private.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.969322 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:26.969373 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.969459 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.969590 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.969677 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "tag:Name", Values: ["masters.complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.complex.example.com"] } ] } I0216 02:06:26.969840 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:26.969881 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:26.969961 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.complex.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.970062 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:26.970102 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.complex.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.complex.example.com"] } ] } I0216 02:06:26.970272 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:26.970364 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:26.971434 46260 executor.go:111] Tasks: 79 done / 123 total; 39 can run I0216 02:06:26.971560 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.971642 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.971670 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:26.971753 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:26.971770 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.971739 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.971811 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.971849 46260 listeners.go:32] DescribeListeners v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.971888 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4"] } I0216 02:06:26.971924 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5"] } I0216 02:06:26.971956 46260 loadbalancers.go:144] DescribeLoadBalancerAttributes v2 { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } I0216 02:06:26.971881 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.972128 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.972293 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:26.972540 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.972692 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.972815 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:26.972945 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.973069 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:26.974468 46260 executor.go:111] Tasks: 118 done / 123 total; 3 can run I0216 02:06:26.974595 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:26.974657 46260 dnsname.go:121] AliasTarget for "api.complex.example.com" is "api-complex-example-com-vd3t5n.amazonaws.com" I0216 02:06:26.974675 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { } I0216 02:06:26.974693 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.974752 46260 executor.go:111] Tasks: 121 done / 123 total; 2 can run I0216 02:06:26.974829 46260 executor.go:111] Tasks: 123 done / 123 total; 0 can run I0216 02:06:26.975065 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:26.975111 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:26.975137 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:26.975159 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:26.975182 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:26.975211 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:26.975231 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:26.975250 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:26.975279 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:26.975305 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:26.975328 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:26.975360 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:26.975382 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:26.975401 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:26.975432 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:26.975454 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:26.975476 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:26.975500 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:26.975527 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:26.975551 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:26.975579 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:26.975604 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:26.975626 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-3"] }] } I0216 02:06:26.975647 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:26.975675 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:26.975694 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:26.975716 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:26.975743 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:26.975768 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:26.977564 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:26.977582 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:26.977604 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977649 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0216 02:06:26.977692 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977737 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977767 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977811 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977837 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977871 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } I0216 02:06:26.977909 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977961 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["complex.example.com"] }] } I0216 02:06:26.977990 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/complex.example.com"] }] } W0216 02:06:26.978026 46260 api.go:53] PageSize not implemented I0216 02:06:26.978039 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:26.978055 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:26.978064 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1"] } I0216 02:06:26.978091 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:26.978111 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:26.978123 46260 tags.go:64] DescribeTags v2 { ResourceArns: [ "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3", "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" ] } I0216 02:06:26.978173 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:26.978194 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:26.978220 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:26.978243 46260 iamrole.go:89] ListRoles: { } I0216 02:06:26.978278 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:26.978312 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:26.978348 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } autoscaling-group:nodes.complex.example.com ok I0216 02:06:26.978706 46260 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tls-complex-example-com-5nursn/4 ok I0216 02:06:26.978747 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:26.978787 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } autoscaling-group:master-us-test-1a.masters.complex.example.com ok I0216 02:06:26.978844 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.complex.example.com" } iam-instance-profile:masters.complex.example.com ok I0216 02:06:26.978770 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-complex-example-com-vd3t5n.amazonaws.com", EvaluateTargetHealth: false, HostedZoneId: "HZ123456" }, Name: "api.complex.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.complex.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.complex.example.com. ok I0216 02:06:26.978758 46260 targetgroups.go:128] DeleteTargetGroup { TargetGroupArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5" } I0216 02:06:26.978872 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } target-group:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/tcp-complex-example-com-vpjolq/5 ok I0216 02:06:26.978940 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.complex.example.com" } iam-instance-profile:nodes.complex.example.com ok I0216 02:06:26.978819 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:26.978973 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0216 02:06:26.978953 46260 loadbalancers.go:196] DeleteLoadBalancer { LoadBalancerArn: "arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1" } load-balancer:arn:aws-test:elasticloadbalancing:us-test-1:000000000000:loadbalancer/net/api-complex-example-com-vd3t5n/1 ok I0216 02:06:26.979006 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok autoscaling-config:lt-2 ok I0216 02:06:26.979069 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:26.979125 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0216 02:06:26.979146 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:26.979177 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [ { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 28000, IpProtocol: "udp", IpRanges: [{ CidrIp: "10.20.30.0/24" }], ToPort: 32767 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 28000, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.2.3.4/32" }], ToPort: 32767 } ] } W0216 02:06:26.979312 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:26.979327 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0216 02:06:26.979347 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-3" } subnet:subnet-3 ok autoscaling-config:lt-1 ok I0216 02:06:26.979384 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:26.979415 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:26.979443 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 8443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 8443 }, { FromPort: 443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 443 }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.1.0.0/16" }], ToPort: 443 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "10.2.0.0/16" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.1/32" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 8443, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-44444444" }], ToPort: 8443 }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 443 }, { FromPort: 22, IpProtocol: "tcp", PrefixListIds: [{ PrefixListId: "pl-66666666" }], ToPort: 22 }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "172.20.0.0/16" }], ToPort: 443 }, { FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "1.1.1.0/24" }], ToPort: 4 } ] } W0216 02:06:26.979633 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:26.979641 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:26.979688 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:26.979719 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.complex.example.com" } I0216 02:06:26.979757 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.complex.example.com I0216 02:06:26.979747 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0216 02:06:26.979770 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.complex.example.com", RoleName: "nodes.complex.example.com" } I0216 02:06:26.979792 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.complex.example.com" } iam-role:nodes.complex.example.com ok I0216 02:06:26.979816 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.complex.example.com" } I0216 02:06:26.979847 46260 iamrole.go:180] ListAttachedRolePolicies: masters.complex.example.com I0216 02:06:26.979855 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.complex.example.com", RoleName: "masters.complex.example.com" } I0216 02:06:26.979870 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.complex.example.com" } iam-role:masters.complex.example.com ok I0216 02:06:26.979906 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:26.979950 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:26.979991 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:26.980177 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleComplex (2.42s) === RUN TestLifecycleExternalLB I0216 02:06:26.980373 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:26.980990 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:26.981081 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:26.981266 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:26.981358 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:26.981408 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:26.981449 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:26.981495 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:26.981529 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:26.981625 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster externallb.example.com I0216 02:06:27.007500 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.009304 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.009369 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.082391 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.166228 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.167941 46260 executor.go:111] Tasks: 0 done / 94 total; 50 can run I0216 02:06:27.168255 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.168321 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.168391 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.168453 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } I0216 02:06:27.169413 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0216 02:06:27.169754 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } W0216 02:06:27.170472 46260 api.go:53] PageSize not implemented W0216 02:06:27.170603 46260 api.go:53] PageSize not implemented I0216 02:06:27.170672 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] } I0216 02:06:27.168474 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:27.168497 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.171264 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } W0216 02:06:27.171441 46260 api.go:53] PageSize not implemented I0216 02:06:27.171382 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.168616 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:27.171554 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:27.171507 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.172189 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.172274 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.172308 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.172366 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.172395 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.172644 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.172821 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.172407 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.173353 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:27.173399 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0216 02:06:27.173475 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:27.173504 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.173658 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:27.173699 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:27.173723 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:27.173755 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.173783 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.173921 46260 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0216 02:06:27.174009 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0216 02:06:27.174038 46260 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0216 02:06:27.174040 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.174109 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.174185 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0216 02:06:27.174210 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:27.174251 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0216 02:06:27.174280 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.174328 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0216 02:06:27.174359 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0216 02:06:27.174410 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:27.174449 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" } ] } I0216 02:06:27.174521 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0216 02:06:27.174601 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:27.174644 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:27.174695 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.174847 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0216 02:06:27.174869 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.externallb.example.com" }, { Key: "KubernetesCluster", Value: "externallb.example.com" }, { Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-1" } I0216 02:06:27.174986 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:27.175040 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:27.175067 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:27.175117 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:27.175150 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.175302 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:27.175346 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0216 02:06:27.175562 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:27.175365 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/externallb.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/externallb.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"externallb.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0216 02:06:27.175613 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.externallb.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:27.175696 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:27.175731 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.177053 46260 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0216 02:06:27.177188 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-4003to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.177318 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.177458 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.177592 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-udp-1to65535-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.177693 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-masters.externallb.example.com" }] }] } I0216 02:06:27.177815 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.177950 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0216 02:06:27.178030 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.178124 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:27.178150 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:27.178175 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.178259 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:27.178338 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" }] }] } I0216 02:06:27.178425 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.178470 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.178535 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.178616 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.178685 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.178772 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.178952 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.externallb.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:27.179081 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:27.179135 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-1to2379-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.179351 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.externallb.example.com-ingress-tcp-2382to4000-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.179579 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.externallb.example.com-ingress-all-0to0-nodes.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.179785 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.externallb.example.com" },{ Key: "KubernetesCluster", Value: "externallb.example.com" },{ Key: "kubernetes.io/cluster/externallb.example.com", Value: "owned" }] }] } I0216 02:06:27.180414 46260 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0216 02:06:27.180607 46260 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0216 02:06:27.180704 46260 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0216 02:06:27.180745 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.180790 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.180845 46260 dns.go:238] Pre-creating DNS records I0216 02:06:27.180872 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.184812 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.186636 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.186699 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.259165 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.351812 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.353600 46260 executor.go:111] Tasks: 0 done / 94 total; 50 can run W0216 02:06:27.353944 46260 api.go:53] PageSize not implemented I0216 02:06:27.353925 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.353867 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } W0216 02:06:27.354082 46260 api.go:53] PageSize not implemented I0216 02:06:27.353978 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.354082 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.externallb.example.com"] }] } W0216 02:06:27.354147 46260 api.go:53] PageSize not implemented I0216 02:06:27.354144 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.354180 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0216 02:06:27.354247 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:27.354215 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:27.354282 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:27.354301 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:27.354323 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.354404 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.354494 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.354501 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.354570 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.externallb.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.354664 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.354650 46260 targetgroups.go:32] DescribeTargetGroups { TargetGroupArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.354730 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1"] } I0216 02:06:27.355895 46260 executor.go:111] Tasks: 50 done / 94 total; 19 can run I0216 02:06:27.356061 46260 iamrole.go:161] ListAttachedRolePolicies: masters.externallb.example.com I0216 02:06:27.356004 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.356088 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.externallb.example.com I0216 02:06:27.356117 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.externallb.example.com"] } ] } I0216 02:06:27.356232 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.356273 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] },{ Name: "tag:Name", Values: ["externallb.example.com"] }] } I0216 02:06:27.356371 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.externallb.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.externallb.example.com"] } ] } I0216 02:06:27.356476 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:27.356517 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.356572 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:27.356618 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.externallb.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.356700 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:27.359553 46260 executor.go:111] Tasks: 69 done / 94 total; 21 can run I0216 02:06:27.359662 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.359764 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.360139 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.360272 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.360453 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.360575 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.360711 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.362122 46260 executor.go:111] Tasks: 90 done / 94 total; 2 can run I0216 02:06:27.362332 46260 executor.go:111] Tasks: 92 done / 94 total; 2 can run I0216 02:06:27.362451 46260 executor.go:111] Tasks: 94 done / 94 total; 0 can run I0216 02:06:27.364781 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:27.364827 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:27.364847 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:27.364866 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:27.364884 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:27.364905 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:27.364924 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:27.364942 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:27.364962 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:27.364983 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:27.365008 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:27.365038 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:27.365068 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:27.365087 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:27.365122 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:27.365155 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:27.365178 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:27.365198 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:27.365218 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:27.365239 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:27.365257 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:27.365275 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:27.365292 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:27.365314 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:27.365335 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:27.365355 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:27.366161 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:27.366180 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:27.366212 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366245 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0216 02:06:27.366278 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366313 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366344 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366381 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366408 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366432 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } I0216 02:06:27.366460 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366495 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["externallb.example.com"] }] } I0216 02:06:27.366521 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/externallb.example.com"] }] } W0216 02:06:27.366554 46260 api.go:53] PageSize not implemented I0216 02:06:27.366568 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:27.366585 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:27.366597 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:27.366610 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:27.366621 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.366654 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.366674 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.366708 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:27.366732 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.366759 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:27.366785 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:27.366804 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:27.367078 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } autoscaling-group:master-us-test-1a.masters.externallb.example.com ok volume:vol-1 ok I0216 02:06:27.367126 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.externallb.example.com ok I0216 02:06:27.367119 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0216 02:06:27.367204 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.externallb.example.com" } iam-instance-profile:masters.externallb.example.com ok I0216 02:06:27.367244 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0216 02:06:27.367307 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.externallb.example.com" } iam-instance-profile:nodes.externallb.example.com ok I0216 02:06:27.367197 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.externallb.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.367162 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 } ] } route53-record:Z1AFAKE1ZON3YO/A/api.externallb.example.com. ok W0216 02:06:27.367350 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:27.367367 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:27.367415 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0216 02:06:27.367479 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:27.367551 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok autoscaling-config:lt-1 ok I0216 02:06:27.367607 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:27.367667 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.367719 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0216 02:06:27.367843 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:27.367865 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-2 ok I0216 02:06:27.367978 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.externallb.example.com" } I0216 02:06:27.368031 46260 iamrole.go:180] ListAttachedRolePolicies: masters.externallb.example.com I0216 02:06:27.368017 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:27.368047 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.externallb.example.com", RoleName: "masters.externallb.example.com" } I0216 02:06:27.368078 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.externallb.example.com" } iam-role:masters.externallb.example.com ok I0216 02:06:27.368110 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.externallb.example.com" } I0216 02:06:27.368151 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.externallb.example.com I0216 02:06:27.368165 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.externallb.example.com", RoleName: "nodes.externallb.example.com" } I0216 02:06:27.368193 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.externallb.example.com" } iam-role:nodes.externallb.example.com ok I0216 02:06:27.368253 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:27.368329 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:27.368404 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:27.368649 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleExternalLB (0.39s) === RUN TestLifecycleSharedSubnet I0216 02:06:27.368931 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:27.369158 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:27.369204 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:27.369253 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:27.369303 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:27.369347 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:27.369376 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:27.369414 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:27.369438 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:27.369487 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster sharedsubnet.example.com I0216 02:06:27.398840 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.401021 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.401081 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.475498 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.554607 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.555708 46260 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0216 02:06:27.556022 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:27.556171 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:27.556213 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:27.556273 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.556223 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.556335 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "nodes.sharedsubnet.example.com" }] } I0216 02:06:27.556337 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.556310 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:27.556484 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } I0216 02:06:27.556549 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] } I0216 02:06:27.556458 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.556677 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.556900 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.557082 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.557260 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:27.557390 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.sharedsubnet.example.com" }, { Key: "KubernetesCluster", Value: "sharedsubnet.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.558908 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:27.560496 46260 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0216 02:06:27.560661 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0216 02:06:27.560783 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0216 02:06:27.561287 46260 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0216 02:06:27.560803 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0216 02:06:27.561336 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0216 02:06:27.561363 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.561422 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.561523 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:27.561574 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:27.561612 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:27.561674 46260 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0216 02:06:27.561673 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.561738 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0216 02:06:27.561877 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.sharedsubnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.561933 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:27.561967 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.562004 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:27.563022 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/sharedsubnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/sharedsubnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"sharedsubnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0216 02:06:27.565116 46260 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0216 02:06:27.565314 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-4003to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0216 02:06:27.565474 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:27.565611 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0216 02:06:27.565748 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" }] }] } I0216 02:06:27.565880 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.sharedsubnet.example.com" }] }] } I0216 02:06:27.566008 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-1to2379-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566094 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566212 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566293 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-ingress-all-0to0-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566430 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566595 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.566669 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-tcp-2382to4000-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.566840 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.566965 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-all-0to0-nodes.sharedsubnet.example.com" }] }] } I0216 02:06:27.567088 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.sharedsubnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.567170 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.sharedsubnet.example.com-ingress-udp-1to65535-masters.sharedsubnet.example.com" },{ Key: "KubernetesCluster", Value: "sharedsubnet.example.com" },{ Key: "kubernetes.io/cluster/sharedsubnet.example.com", Value: "owned" }] }] } I0216 02:06:27.567271 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.567354 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.569251 46260 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0216 02:06:27.569482 46260 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0216 02:06:27.569583 46260 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0216 02:06:27.569624 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.569662 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.569725 46260 dns.go:238] Pre-creating DNS records I0216 02:06:27.569753 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.576246 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.578105 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.578169 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.653699 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.725592 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.727100 46260 executor.go:111] Tasks: 0 done / 80 total; 42 can run I0216 02:06:27.727466 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:27.727499 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:27.727561 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:27.727582 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:27.727612 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.727745 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.sharedsubnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.727859 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.727901 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.sharedsubnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.729408 46260 executor.go:111] Tasks: 42 done / 80 total; 16 can run I0216 02:06:27.729560 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0216 02:06:27.729635 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0216 02:06:27.729693 46260 iamrole.go:161] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0216 02:06:27.729710 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.sharedsubnet.example.com"] } ] } I0216 02:06:27.729899 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.729954 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:27.730046 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.sharedsubnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.sharedsubnet.example.com"] } ] } I0216 02:06:27.730145 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:27.732401 46260 executor.go:111] Tasks: 58 done / 80 total; 18 can run I0216 02:06:27.732592 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.732687 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.732898 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.733088 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:27.735351 46260 executor.go:111] Tasks: 76 done / 80 total; 2 can run I0216 02:06:27.735573 46260 executor.go:111] Tasks: 78 done / 80 total; 2 can run I0216 02:06:27.735706 46260 executor.go:111] Tasks: 80 done / 80 total; 0 can run I0216 02:06:27.737619 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:27.737679 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:27.737699 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:27.737718 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:27.737740 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:27.737758 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:27.737777 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:27.737797 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:27.737818 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:27.737842 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:27.737867 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:27.737886 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:27.737903 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:27.737920 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:27.737939 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:27.737959 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:27.737980 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:27.737998 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:27.738016 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:27.738037 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:27.738057 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:27.738955 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:27.738976 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:27.739004 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739037 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0216 02:06:27.739072 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739107 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739132 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739157 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739194 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739217 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } I0216 02:06:27.739241 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739269 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["sharedsubnet.example.com"] }] } I0216 02:06:27.739290 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/sharedsubnet.example.com"] }] } W0216 02:06:27.739319 46260 api.go:53] PageSize not implemented I0216 02:06:27.739333 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:27.739351 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:27.739362 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:27.739377 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:27.739389 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:27.739422 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.739442 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.739476 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:27.739500 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.739534 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:27.739562 46260 routetable.go:53] DescribeRouteTables: { } autoscaling-group:nodes.sharedsubnet.example.com ok I0216 02:06:27.739825 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0216 02:06:27.739838 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } I0216 02:06:27.739876 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.sharedsubnet.example.com" } iam-instance-profile:masters.sharedsubnet.example.com ok I0216 02:06:27.739908 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:master-us-test-1a.masters.sharedsubnet.example.com ok volume:vol-1 ok I0216 02:06:27.739922 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.739969 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.sharedsubnet.example.com" } iam-instance-profile:nodes.sharedsubnet.example.com ok I0216 02:06:27.739943 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }] } W0216 02:06:27.740035 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:27.740046 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok autoscaling-config:lt-1 ok I0216 02:06:27.739933 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.sharedsubnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.internal.sharedsubnet.example.com. ok I0216 02:06:27.740117 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:27.740175 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-2 ok I0216 02:06:27.740271 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.740322 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0216 02:06:27.740529 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:27.740545 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:27.740614 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.740684 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.sharedsubnet.example.com I0216 02:06:27.740701 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.sharedsubnet.example.com", RoleName: "nodes.sharedsubnet.example.com" } I0216 02:06:27.740735 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.sharedsubnet.example.com" } iam-role:nodes.sharedsubnet.example.com ok I0216 02:06:27.740770 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.sharedsubnet.example.com" } I0216 02:06:27.740822 46260 iamrole.go:180] ListAttachedRolePolicies: masters.sharedsubnet.example.com I0216 02:06:27.740837 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.sharedsubnet.example.com", RoleName: "masters.sharedsubnet.example.com" } I0216 02:06:27.740866 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.sharedsubnet.example.com" } iam-role:masters.sharedsubnet.example.com ok I0216 02:06:27.741110 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleSharedSubnet (0.37s) === RUN TestLifecyclePrivateSharedSubnet I0216 02:06:27.741446 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:27.741692 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:27.741738 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:27.741779 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:27.741825 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:27.741864 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:27.741890 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:27.741928 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:27.741954 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:27.742007 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-subnet.example.com I0216 02:06:27.767798 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.769733 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.769796 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.769835 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.843797 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.909594 46260 iamrole.go:89] ListRoles: { } I0216 02:06:27.910913 46260 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0216 02:06:27.911195 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:27.911250 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] }] } I0216 02:06:27.911379 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.911396 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } I0216 02:06:27.911486 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0216 02:06:27.911438 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.911612 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:27.911682 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:27.911840 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:27.911943 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:27.911978 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:27.911929 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } I0216 02:06:27.912015 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:27.912039 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0216 02:06:27.912129 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } I0216 02:06:27.912159 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-subnet.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:27.912243 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:27.912205 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", Tags: [{ Key: "Name", Value: "bastions.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] } I0216 02:06:27.912271 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-subnet.example.com" }] }] } I0216 02:06:27.913592 46260 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0216 02:06:27.913702 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0216 02:06:27.913826 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:27.913896 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:27.913888 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0216 02:06:27.913945 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0216 02:06:27.914123 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0216 02:06:27.914173 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.914198 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0216 02:06:27.914279 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:27.914282 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0216 02:06:27.914315 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:27.914334 46260 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0216 02:06:27.914362 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0216 02:06:27.914340 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0216 02:06:27.914428 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0216 02:06:27.914452 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.914508 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:27.914502 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0216 02:06:27.914534 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:27.914555 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0216 02:06:27.914664 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.914721 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:27.914746 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:27.914766 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0216 02:06:27.914876 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.914926 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:27.914949 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:27.914972 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0216 02:06:27.915081 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:27.915109 46260 tags.go:42] CreateTags { Resources: ["subnet-12345678"], Tags: [ { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] } I0216 02:06:27.915159 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0216 02:06:27.915225 46260 tags.go:42] CreateTags { Resources: ["subnet-abcdef"], Tags: [ { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "shared" }, { Key: "SubnetType", Value: "Utility" } ] } I0216 02:06:27.915316 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-subnet.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:27.915391 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:27.915424 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:27.915609 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-subnet.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-subnet.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-subnet.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0216 02:06:27.917850 46260 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0216 02:06:27.918141 46260 api.go:53] PageSize not implemented I0216 02:06:27.918043 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } W0216 02:06:27.918183 46260 api.go:53] PageSize not implemented I0216 02:06:27.918286 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0216 02:06:27.918286 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0216 02:06:27.918378 46260 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-subnet.example.com" }] } I0216 02:06:27.918442 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0216 02:06:27.918440 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:27.918473 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0216 02:06:27.918522 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0216 02:06:27.918580 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-udp-1to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.918603 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0216 02:06:27.918677 46260 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-subnet.example.com" }] } I0216 02:06:27.918736 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0216 02:06:27.918723 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0216 02:06:27.918764 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0216 02:06:27.918809 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0216 02:06:27.918849 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-ingress-tcp-22to22-bastion.private-shared-subnet.example.com" }] }] } I0216 02:06:27.918932 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.919077 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.919148 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.919214 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.919518 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-subnet.example.com" }] }] } I0216 02:06:27.919726 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-masters.private-shared-subnet.example.com" }] }] } I0216 02:06:27.919885 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-2382to4000-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.920038 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-ingress-all-0to0-nodes.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.920181 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0216 02:06:27.920292 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.920451 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-ingress-tcp-22to22-nodes.private-shared-subnet.example.com" }] }] } I0216 02:06:27.920606 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-subnet.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0216 02:06:27.920772 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.920876 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.920990 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:27.921135 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }] } I0216 02:06:27.921241 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:27.921354 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:27.921477 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-1to2379-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" }] }] } I0216 02:06:27.921616 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-api-elb.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.921744 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.921875 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.921960 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.922034 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-subnet.example.com-ingress-tcp-4003to65535-masters.private-shared-subnet.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-subnet.example.com" },{ Key: "kubernetes.io/cluster/private-shared-subnet.example.com", Value: "owned" }] }] } I0216 02:06:27.924139 46260 executor.go:111] Tasks: 100 done / 107 total; 4 can run I0216 02:06:27.924279 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:27.924490 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:27.924655 46260 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0216 02:06:27.924839 46260 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0216 02:06:27.924904 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:27.924939 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.925007 46260 dns.go:238] Pre-creating DNS records I0216 02:06:27.925034 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:27.930149 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:27.932043 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.932102 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:27.932141 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.004959 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.094448 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.096882 46260 executor.go:111] Tasks: 0 done / 107 total; 45 can run I0216 02:06:28.097237 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.097368 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:28.097526 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-subnet.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.097647 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:28.097711 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:28.097741 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:28.097790 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.097874 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-subnet.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.099172 46260 executor.go:111] Tasks: 45 done / 107 total; 24 can run I0216 02:06:28.099270 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0216 02:06:28.099399 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0216 02:06:28.099329 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-subnet.example.com"] } ] } I0216 02:06:28.099439 46260 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0216 02:06:28.099485 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-abcdef"] } I0216 02:06:28.099610 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-subnet.example.com"] } ] } I0216 02:06:28.100411 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.100754 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-12345678"] } I0216 02:06:28.100880 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-subnet.example.com"] } ] } I0216 02:06:28.101077 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:28.101210 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:28.101285 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-subnet.example.com"] } ] } I0216 02:06:28.101451 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-subnet.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-subnet.example.com"] } ] } I0216 02:06:28.101712 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.101893 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:28.102057 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:28.106048 46260 executor.go:111] Tasks: 69 done / 107 total; 31 can run W0216 02:06:28.106176 46260 api.go:53] PageSize not implemented I0216 02:06:28.106208 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q","api-private-shared-subnet-n2f8ak"] } I0216 02:06:28.106220 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.106324 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } I0216 02:06:28.106363 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } I0216 02:06:28.106360 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } W0216 02:06:28.106450 46260 api.go:53] PageSize not implemented I0216 02:06:28.106484 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.106470 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0216 02:06:28.106544 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-su-5ol32q"] } I0216 02:06:28.106577 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } I0216 02:06:28.106787 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.106875 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.107043 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.108441 46260 executor.go:111] Tasks: 100 done / 107 total; 4 can run W0216 02:06:28.108514 46260 api.go:53] PageSize not implemented I0216 02:06:28.108529 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0216 02:06:28.108537 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:28.108614 46260 dnsname.go:121] AliasTarget for "api.private-shared-subnet.example.com" is "api-private-shared-subnet-n2f8ak.elb.cloudmock.com" I0216 02:06:28.108638 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak"] } W0216 02:06:28.108690 46260 api.go:53] PageSize not implemented I0216 02:06:28.108707 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0216 02:06:28.108865 46260 executor.go:111] Tasks: 104 done / 107 total; 3 can run I0216 02:06:28.108996 46260 executor.go:111] Tasks: 107 done / 107 total; 0 can run I0216 02:06:28.113027 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:28.113091 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:28.113129 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:28.113168 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:28.113202 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.113238 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:28.113272 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:28.113307 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:28.113347 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:28.113386 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:28.113426 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0216 02:06:28.113471 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:28.113507 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:28.113540 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:28.113575 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.113622 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:28.113663 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:28.113702 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:28.113741 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:28.113780 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:28.113820 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:28.113860 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:28.113896 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:28.113936 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:28.113996 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:28.116055 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:28.116084 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:28.116126 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116190 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0216 02:06:28.116270 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116330 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116374 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116421 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116465 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116501 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } I0216 02:06:28.116545 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116592 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-subnet.example.com"] }] } I0216 02:06:28.116632 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-subnet.example.com"] }] } W0216 02:06:28.116678 46260 api.go:53] PageSize not implemented I0216 02:06:28.116698 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-subnet-n2f8ak","bastion-private-shared-su-5ol32q"] } I0216 02:06:28.116752 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:28.116780 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:28.116800 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:28.116824 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:28.116843 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:28.116894 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.116922 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.116969 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:28.117014 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.117059 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:28.117099 46260 routetable.go:53] DescribeRouteTables: { } autoscaling-config:lt-3 ok autoscaling-config:lt-2 ok I0216 02:06:28.117671 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } autoscaling-group:bastion.private-shared-subnet.example.com ok I0216 02:06:28.117736 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-subnet.example.com" } iam-instance-profile:masters.private-shared-subnet.example.com ok I0216 02:06:28.117710 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-su-5ol32q" } autoscaling-group:nodes.private-shared-subnet.example.com ok I0216 02:06:28.117682 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } load-balancer:bastion-private-shared-su-5ol32q ok autoscaling-group:master-us-test-1a.masters.private-shared-subnet.example.com ok I0216 02:06:28.117792 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-subnet-n2f8ak" } load-balancer:api-private-shared-subnet-n2f8ak ok I0216 02:06:28.117761 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0216 02:06:28.117835 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0216 02:06:28.117875 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-subnet.example.com" } iam-instance-profile:bastions.private-shared-subnet.example.com ok I0216 02:06:28.117814 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } I0216 02:06:28.117738 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-subnet-n2f8ak.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-subnet.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-subnet.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.private-shared-subnet.example.com ok W0216 02:06:28.117921 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.117902 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-subnet.example.com" } iam-instance-profile:nodes.private-shared-subnet.example.com ok I0216 02:06:28.117961 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok autoscaling-config:lt-1 ok I0216 02:06:28.118000 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:28.118056 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } W0216 02:06:28.118127 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.118140 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0216 02:06:28.118162 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:28.118196 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:28.118225 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok I0216 02:06:28.118258 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.118284 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [ { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-2" }] } ] } W0216 02:06:28.118396 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.118405 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:28.118445 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:28.118459 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-subnet.example.com" } I0216 02:06:28.118524 46260 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-subnet.example.com I0216 02:06:28.118474 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0216 02:06:28.118554 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.118541 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-subnet.example.com", RoleName: "masters.private-shared-subnet.example.com" } I0216 02:06:28.118570 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } I0216 02:06:28.118576 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-subnet.example.com" } security-group:sg-3 ok iam-role:masters.private-shared-subnet.example.com ok I0216 02:06:28.118602 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:28.118610 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-subnet.example.com" } I0216 02:06:28.118658 46260 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-subnet.example.com I0216 02:06:28.118674 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-subnet.example.com", RoleName: "bastions.private-shared-subnet.example.com" } I0216 02:06:28.118707 46260 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-subnet.example.com" } iam-role:bastions.private-shared-subnet.example.com ok I0216 02:06:28.118648 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0216 02:06:28.118742 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.118755 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0216 02:06:28.118740 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-subnet.example.com" } I0216 02:06:28.118788 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-subnet.example.com I0216 02:06:28.118800 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-subnet.example.com", RoleName: "nodes.private-shared-subnet.example.com" } I0216 02:06:28.118820 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-subnet.example.com" } iam-role:nodes.private-shared-subnet.example.com ok I0216 02:06:28.119025 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedSubnet (0.38s) === RUN TestLifecyclePrivateSharedIP I0216 02:06:28.119272 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:28.119471 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:28.119505 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:28.119532 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:28.119563 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:28.119592 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:28.119610 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:28.119634 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:28.119650 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:28.119683 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster private-shared-ip.example.com I0216 02:06:28.136606 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:28.138552 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.138613 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.138650 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.211978 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.292230 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.294457 46260 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0216 02:06:28.294653 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] }] } I0216 02:06:28.294873 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.294980 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0216 02:06:28.294849 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-main.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:28.295093 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:28.295078 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:28.295120 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } I0216 02:06:28.295152 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.295305 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.295310 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:28.295402 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:28.295381 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastions.private-shared-ip.example.com" }] } I0216 02:06:28.295438 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.295452 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", Tags: [{ Key: "Name", Value: "masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] } I0216 02:06:28.295474 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.295550 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:28.295647 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:28.295682 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:28.295718 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.296790 46260 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0216 02:06:28.296885 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.296958 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0216 02:06:28.297002 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0216 02:06:28.297087 46260 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0216 02:06:28.297117 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0216 02:06:28.297112 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] }] } I0216 02:06:28.297083 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kops.k8s.io/instance-group/nodes", Value: "true" }, { Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Private" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" } ] }], VpcId: "vpc-12345678" } I0216 02:06:28.297182 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["private-us-test-1a"] }] } I0216 02:06:28.297218 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0216 02:06:28.297266 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] }], VpcId: "vpc-12345678" } I0216 02:06:28.297342 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.297273 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0216 02:06:28.297391 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:DescribeRegions\",\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.297459 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.297500 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.297564 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.297547 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.297593 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.297649 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:28.297733 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" } ] }], VpcId: "vpc-12345678" } I0216 02:06:28.297759 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:28.297837 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.297870 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.297907 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.298022 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.4.0/22", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Utility" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/bastion", Value: "true" }, { Key: "Name", Value: "utility-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" } ] }], VpcId: "vpc-12345678" } I0216 02:06:28.298047 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0216 02:06:28.298257 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "masters.private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0216 02:06:28.298350 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0216 02:06:28.298463 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:28.298548 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for api ELB", GroupName: "api-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:28.298635 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0216 02:06:28.298773 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:28.298820 46260 tags.go:42] CreateTags { Resources: ["rtb-2"], Tags: [ { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "private-shared-ip.example.com" } ] } I0216 02:06:28.298893 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:28.298936 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "Name", Value: "private-us-test-1a.private-shared-ip.example.com" }, { Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }, { Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "private-us-test-1a" } ] } I0216 02:06:28.298794 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/private-shared-ip.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/private-shared-ip.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"private-shared-ip.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0216 02:06:28.299007 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:28.299081 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.299111 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion ELB", GroupName: "bastion-elb.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }], VpcId: "vpc-12345678" } I0216 02:06:28.299169 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:28.299196 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:28.299216 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:28.299249 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:28.299270 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for bastion", GroupName: "bastion.private-shared-ip.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }], VpcId: "vpc-12345678" } I0216 02:06:28.299320 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:28.299347 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:28.299372 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:28.299396 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:28.301936 46260 executor.go:111] Tasks: 71 done / 116 total; 36 can run I0216 02:06:28.302115 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } W0216 02:06:28.302208 46260 api.go:53] PageSize not implemented I0216 02:06:28.302228 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0216 02:06:28.302308 46260 api.go:53] PageSize not implemented I0216 02:06:28.302332 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0216 02:06:28.302330 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0216 02:06:28.302393 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0216 02:06:28.302372 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.302462 46260 tags.go:74] AddTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"], Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "api.private-shared-ip.example.com" }] } I0216 02:06:28.302504 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0216 02:06:28.302526 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "SSL:443", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0216 02:06:28.302562 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: true, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0216 02:06:28.302543 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-::/0" }] }] } I0216 02:06:28.302609 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0216 02:06:28.302628 46260 tags.go:74] AddTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] } I0216 02:06:28.302660 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0216 02:06:28.302676 46260 healthcheck.go:31] ConfigureHealthCheck: { HealthCheck: { HealthyThreshold: 2, Interval: 10, Target: "TCP:22", Timeout: 5, UnhealthyThreshold: 2 }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0216 02:06:28.302670 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }] } I0216 02:06:28.302702 46260 attributes.go:31] ModifyLoadBalancerAttributes: { LoadBalancerAttributes: { AccessLog: { Enabled: false }, ConnectionDraining: { Enabled: false, Timeout: 300 }, ConnectionSettings: { IdleTimeout: 300 }, CrossZoneLoadBalancing: { Enabled: false } }, LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0216 02:06:28.302743 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-4003to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.302876 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 }] } I0216 02:06:28.302984 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-4", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:28.303109 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.303210 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:28.303303 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-2"] }] } I0216 02:06:28.303328 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-2", SubnetId: "subnet-2" } I0216 02:06:28.303403 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.303497 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-bastion-elb.private-shared-ip.example.com" }] }] } I0216 02:06:28.303569 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-ingress-tcp-22to22-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.303664 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.303746 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:28.303824 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0216 02:06:28.303901 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-1to2379-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.304025 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-api-elb.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304104 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304185 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:28.304212 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:28.304232 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-bastion-elb.private-shared-ip.example.com-ingress-tcp-22to22-bastion.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.304310 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304383 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304462 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.304526 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.304596 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.304721 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-udp-1to65535-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304812 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-5", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-bastion.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.304900 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.304974 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-3", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-api-elb.private-shared-ip.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.305054 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.private-shared-ip.example.com-ingress-all-0to0-nodes.private-shared-ip.example.com" }] }] } I0216 02:06:28.305140 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-1", RouteTableId: "rtb-2" } I0216 02:06:28.305175 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.305246 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.305307 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.305395 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.305455 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.private-shared-ip.example.com-ingress-tcp-2382to4000-masters.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" }] }] } I0216 02:06:28.305557 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.305701 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.305744 46260 tags.go:42] CreateTags { Resources: ["eipalloc-12345678"], Tags: [{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "shared" }] } I0216 02:06:28.306814 46260 executor.go:111] Tasks: 107 done / 116 total; 5 can run I0216 02:06:28.306910 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.306961 46260 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedNatgateway"] },{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:28.306941 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:28.307049 46260 natgateway.go:74] CreateNatGateway: { AllocationId: "eipalloc-12345678", SubnetId: "subnet-2", TagSpecifications: [{ ResourceType: "natgateway", Tags: [{ Key: "Name", Value: "us-test-1a.private-shared-ip.example.com" },{ Key: "KubernetesCluster", Value: "private-shared-ip.example.com" },{ Key: "kubernetes.io/cluster/private-shared-ip.example.com", Value: "owned" }] }] } I0216 02:06:28.307109 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:28.307072 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "UPSERT", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } }] }, HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:28.307134 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:28.307160 46260 tags.go:42] CreateTags { Resources: ["subnet-2"], Tags: [{ Key: "AssociatedNatgateway", Value: "nat-1" }] } I0216 02:06:28.307199 46260 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0216 02:06:28.307299 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.307409 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: "nat-1", RouteTableId: "rtb-1" } I0216 02:06:28.307457 46260 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0216 02:06:28.307486 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.307506 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.307538 46260 dns.go:238] Pre-creating DNS records I0216 02:06:28.307553 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.314155 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.21.14 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.21.14 ********************************************************************************* I0216 02:06:28.316041 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.316100 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.316136 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.388143 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.487763 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.490109 46260 executor.go:111] Tasks: 0 done / 116 total; 45 can run I0216 02:06:28.490430 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:28.490407 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.490651 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.490729 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.private-shared-ip.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.490855 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-12345678"] } I0216 02:06:28.490917 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-12345678" } I0216 02:06:28.490938 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-12345678" } I0216 02:06:28.490965 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.492599 46260 executor.go:111] Tasks: 45 done / 116 total; 26 can run I0216 02:06:28.492701 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["utility-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.492825 46260 iamrole.go:161] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0216 02:06:28.492866 46260 iamrole.go:161] ListAttachedRolePolicies: masters.private-shared-ip.example.com I0216 02:06:28.492910 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0216 02:06:28.492829 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] },{ Name: "tag:Name", Values: ["us-test-1a.private-shared-ip.example.com"] }] } I0216 02:06:28.492984 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["nodes.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["nodes.private-shared-ip.example.com"] } ] } I0216 02:06:28.494083 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.494141 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "tag:Name", Values: ["bastion.private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion.private-shared-ip.example.com"] } ] } I0216 02:06:28.494341 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-us-test-1a.private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.494505 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["api-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["api-elb.private-shared-ip.example.com"] } ] } I0216 02:06:28.494643 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-12345678"] }] } I0216 02:06:28.494761 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["bastion-elb.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["bastion-elb.private-shared-ip.example.com"] } ] } I0216 02:06:28.495025 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.private-shared-ip.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }, { Name: "vpc-id", Values: ["vpc-12345678"] }, { Name: "group-name", Values: ["masters.private-shared-ip.example.com"] } ] } I0216 02:06:28.495164 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-2"] } I0216 02:06:28.495247 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["private-shared-ip.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.495375 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.495591 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } I0216 02:06:28.495656 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:28.496145 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:28.496196 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.500399 46260 executor.go:111] Tasks: 71 done / 116 total; 36 can run W0216 02:06:28.500588 46260 api.go:53] PageSize not implemented I0216 02:06:28.500543 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:28.500615 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0216 02:06:28.500712 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["bastion-private-shared-ip-eepmph"] } I0216 02:06:28.500691 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } W0216 02:06:28.500748 46260 api.go:53] PageSize not implemented I0216 02:06:28.500783 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0216 02:06:28.500896 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } I0216 02:06:28.500934 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } I0216 02:06:28.500974 46260 attributes.go:52] DescribeLoadBalancerAttributes: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } I0216 02:06:28.500882 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501078 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501284 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501437 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501569 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501688 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.14-debian-stretch-amd64-hvm-ebs-2019-08-16"] }], Owners: ["383156758163"] } I0216 02:06:28.501753 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } I0216 02:06:28.501861 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.502059 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.502158 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2"] } I0216 02:06:28.502906 46260 executor.go:111] Tasks: 107 done / 116 total; 5 can run W0216 02:06:28.502971 46260 api.go:53] PageSize not implemented I0216 02:06:28.502983 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0216 02:06:28.502976 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "Z1AFAKE1ZON3YO" } I0216 02:06:28.502992 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.503046 46260 dnsname.go:121] AliasTarget for "api.private-shared-ip.example.com" is "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com" I0216 02:06:28.503066 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:28.503074 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj"] } W0216 02:06:28.503122 46260 api.go:53] PageSize not implemented I0216 02:06:28.503139 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0216 02:06:28.503278 46260 executor.go:111] Tasks: 112 done / 116 total; 4 can run I0216 02:06:28.503347 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.503462 46260 executor.go:111] Tasks: 116 done / 116 total; 0 can run I0216 02:06:28.507454 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:28.507517 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:28.507557 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:28.507596 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:28.507633 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.507672 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:28.507712 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:28.507749 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:28.507791 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:28.507836 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:28.507881 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-3"] }] } I0216 02:06:28.507936 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-1"] }] } I0216 02:06:28.507979 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:28.508016 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:28.508052 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:28.508091 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:28.508127 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-2"] }] } I0216 02:06:28.508165 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.508204 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:28.508243 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-3"] }] } I0216 02:06:28.508282 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-4"] }] } I0216 02:06:28.508326 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-5"] }] } I0216 02:06:28.508365 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.508406 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:28.508443 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-2"] }] } I0216 02:06:28.508484 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:28.508521 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:28.508567 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:28.508648 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:28.508726 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:28.510592 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:28.510619 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:28.510671 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.510735 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0216 02:06:28.510815 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.510872 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.510918 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.510962 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.511012 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.511059 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } I0216 02:06:28.511115 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.511174 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:28.511203 46260 natgateway.go:117] DescribeNatGateways: { } I0216 02:06:28.511245 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["private-shared-ip.example.com"] }] } I0216 02:06:28.511284 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/private-shared-ip.example.com"] }] } W0216 02:06:28.511331 46260 api.go:53] PageSize not implemented I0216 02:06:28.511350 46260 tags.go:31] DescribeTags { LoadBalancerNames: ["api-private-shared-ip-exa-ohatqj","bastion-private-shared-ip-eepmph"] } I0216 02:06:28.511401 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:28.511430 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:28.511448 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:28.511473 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:28.511489 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3"] } I0216 02:06:28.511539 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.511568 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.511611 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:28.511650 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.511701 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:28.511745 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:28.511780 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-2","rtb-1"] } I0216 02:06:28.511818 46260 natgateway.go:117] DescribeNatGateways: { NatGatewayIds: ["nat-1"] } I0216 02:06:28.511850 46260 address.go:120] DescribeAddresses: { AllocationIds: ["eipalloc-12345678"] } autoscaling-group:master-us-test-1a.masters.private-shared-ip.example.com ok I0216 02:06:28.512464 46260 natgateway.go:186] DeleteNatGateway: { NatGatewayId: "nat-1" } autoscaling-group:bastion.private-shared-ip.example.com ok autoscaling-group:nodes.private-shared-ip.example.com ok I0216 02:06:28.512501 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.512542 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "bastion-private-shared-ip-eepmph" } load-balancer:bastion-private-shared-ip-eepmph ok I0216 02:06:28.512584 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "bastions.private-shared-ip.example.com" } iam-instance-profile:bastions.private-shared-ip.example.com ok I0216 02:06:28.512603 46260 api.go:149] DeleteLoadBalancer: { LoadBalancerName: "api-private-shared-ip-exa-ohatqj" } nat-gateway:nat-1 ok I0216 02:06:28.512534 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-5"] } load-balancer:api-private-shared-ip-exa-ohatqj ok I0216 02:06:28.512609 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0216 02:06:28.512672 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.private-shared-ip.example.com" } iam-instance-profile:masters.private-shared-ip.example.com ok I0216 02:06:28.512651 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-5", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-4" }] }] } W0216 02:06:28.512717 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions autoscaling-config:lt-3 ok I0216 02:06:28.512711 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.512747 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.private-shared-ip.example.com" } iam-instance-profile:nodes.private-shared-ip.example.com ok I0216 02:06:28.512728 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-5" } security-group:sg-5 ok I0216 02:06:28.512774 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.512615 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { AliasTarget: { DNSName: "api-private-shared-ip-exa-ohatqj.elb.cloudmock.com", EvaluateTargetHealth: false, HostedZoneId: "FAKEZONE-CLOUDMOCK-ELB" }, Name: "api.private-shared-ip.example.com", Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.private-shared-ip.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/kops-controller.internal.private-shared-ip.example.com. ok I0216 02:06:28.512809 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 443, IpProtocol: "tcp", ToPort: 443, UserIdGroupPairs: [{ GroupId: "sg-3" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] } ] } W0216 02:06:28.512944 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.512958 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:28.512980 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.513008 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", ToPort: 22, UserIdGroupPairs: [{ GroupId: "sg-5" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } W0216 02:06:28.513085 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.513094 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:28.513113 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok autoscaling-config:lt-2 ok I0216 02:06:28.513161 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok I0216 02:06:28.513202 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:28.513257 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-2" } subnet:subnet-2 ok I0216 02:06:28.513289 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.private-shared-ip.example.com" } I0216 02:06:28.513291 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } I0216 02:06:28.513340 46260 iamrole.go:180] ListAttachedRolePolicies: masters.private-shared-ip.example.com subnet:subnet-1 ok I0216 02:06:28.513350 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.private-shared-ip.example.com", RoleName: "masters.private-shared-ip.example.com" } I0216 02:06:28.513374 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.private-shared-ip.example.com" } iam-role:masters.private-shared-ip.example.com ok I0216 02:06:28.513357 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-4"] } I0216 02:06:28.513395 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.513421 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.private-shared-ip.example.com I0216 02:06:28.513430 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.private-shared-ip.example.com", RoleName: "nodes.private-shared-ip.example.com" } I0216 02:06:28.513449 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.private-shared-ip.example.com" } iam-role:nodes.private-shared-ip.example.com ok I0216 02:06:28.513410 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-4", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }] } W0216 02:06:28.513498 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.513514 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-4" } security-group:sg-4 ok I0216 02:06:28.513468 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.513568 46260 iamrole.go:180] ListAttachedRolePolicies: bastions.private-shared-ip.example.com I0216 02:06:28.513549 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-3"] } I0216 02:06:28.513578 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "bastions.private-shared-ip.example.com", RoleName: "bastions.private-shared-ip.example.com" } I0216 02:06:28.513599 46260 iamrole.go:137] DeleteRole: { RoleName: "bastions.private-shared-ip.example.com" } iam-role:bastions.private-shared-ip.example.com ok I0216 02:06:28.513597 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-3", IpPermissions: [{ FromPort: 3, IpProtocol: "icmp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 4 },{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }] } W0216 02:06:28.513705 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.513718 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-3" } security-group:sg-3 ok I0216 02:06:28.513785 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:28.513842 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-2" } route-table:rtb-2 ok I0216 02:06:28.514134 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecyclePrivateSharedIP (0.40s) === RUN TestLifecycleNodeTerminationHandlerQueueProcessor I0216 02:06:28.514453 46260 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:28.514719 46260 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:28.514764 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:28.514807 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:28.514855 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:28.514898 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:28.514933 46260 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:28.514974 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:28.515001 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:28.515059 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=true lifecycle_integration_test.go:194: running lifecycle test for cluster nthsqsresources.longclustername.example.com I0216 02:06:28.542680 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0216 02:06:28.545820 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.545916 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.619148 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.706531 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.707708 46260 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0216 02:06:28.707921 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:28.707996 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.708220 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.708348 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.708439 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] } I0216 02:06:28.708436 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "Name", Value: "us-test-1a.etcd-main.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/main", Value: "us-test-1a/us-test-1a" }, { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:28.708615 46260 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.708687 46260 iaminstanceprofile.go:100] CreateInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" }] } I0216 02:06:28.708642 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:28.708785 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.708961 46260 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.708970 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.709171 46260 volumes.go:33] CreateVolume: { AvailabilityZone: "us-test-1a", Encrypted: false, Iops: 3000, Size: 20, TagSpecifications: [{ ResourceType: "volume", Tags: [ { Key: "k8s.io/role/master", Value: "1" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "Name", Value: "us-test-1a.etcd-events.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "k8s.io/etcd/events", Value: "us-test-1a/us-test-1a" } ] }], Throughput: 125, VolumeType: "gp3" } I0216 02:06:28.709296 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:28.709342 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.709399 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.709546 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsSupport: { Value: true }, VpcId: "vpc-1" } I0216 02:06:28.709593 46260 vpcs.go:190] ModifyVpcAttribute: { EnableDnsHostnames: { Value: true }, VpcId: "vpc-1" } I0216 02:06:28.709638 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.709692 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.709724 46260 keypairs.go:50] ImportKeyPair: { KeyName: "kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57", PublicKeyMaterial: len 212, TagSpecifications: [{ ResourceType: "key-pair", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.709871 46260 dhcpoptions.go:125] CreateDhcpOptions: { DhcpConfigurations: [{ Key: "domain-name-servers", Values: ["AmazonProvidedDNS"] },{ Key: "domain-name", Values: ["us-test-1.compute.internal"] }], TagSpecifications: [{ ResourceType: "dhcp-options", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.709943 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:28.710101 46260 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0216 02:06:28.710211 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0216 02:06:28.710179 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.710247 46260 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0216 02:06:28.710245 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0216 02:06:28.710285 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.710427 46260 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.710432 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.710589 46260 internetgateways.go:66] CreateInternetGateway: { TagSpecifications: [{ ResourceType: "internet-gateway", Tags: [{ Key: "Name", Value: "nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.710687 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:28.710729 46260 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:28.710777 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.710842 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] },{ Name: "tag:kubernetes.io/kops/role", Values: ["public"] }] } I0216 02:06:28.710893 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/addons/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/cluster-completed.spec\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/igconfig/node/*\",\n \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/secrets/dockerconfig\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingInstances\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeRegions\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:GenerateRandom\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.710916 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.710947 46260 routetable.go:106] CreateRouteTable: { TagSpecifications: [{ ResourceType: "route-table", Tags: [ { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" } ] }], VpcId: "vpc-1" } I0216 02:06:28.711059 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:28.711109 46260 tags.go:42] CreateTags { Resources: ["rtb-1"], Tags: [ { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "kubernetes.io/kops/role", Value: "public" }, { Key: "Name", Value: "nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" } ] } I0216 02:06:28.711116 46260 subnets.go:110] CreateSubnet: { AvailabilityZone: "us-test-1a", CidrBlock: "172.20.32.0/19", TagSpecifications: [{ ResourceType: "subnet", Tags: [ { Key: "Name", Value: "us-test-1a.nthsqsresources.longclustername.example.com" }, { Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }, { Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }, { Key: "SubnetType", Value: "Public" }, { Key: "kubernetes.io/role/elb", Value: "1" }, { Key: "kubernetes.io/role/internal-elb", Value: "1" }, { Key: "kops.k8s.io/instance-group/master-us-test-1a", Value: "true" }, { Key: "kops.k8s.io/instance-group/nodes", Value: "true" } ] }], VpcId: "vpc-1" } I0216 02:06:28.711201 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0216 02:06:28.711356 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for nodes", GroupName: "nodes.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }], VpcId: "vpc-1" } I0216 02:06:28.711434 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.711493 46260 securitygroups.go:40] CreateSecurityGroup: { Description: "Security group for masters", GroupName: "masters.nthsqsresources.longclustername.example.com", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "masters.nthsqsresources.longclustername.example.com" }] }], VpcId: "vpc-1" } I0216 02:06:28.712270 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.712368 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.712526 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.712610 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.712681 46260 dhcpoptions.go:91] AssociateDhcpOptions: { DhcpOptionsId: "dopt-1", VpcId: "vpc-1" } I0216 02:06:28.712727 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:28.712799 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.712488 46260 iamrolepolicy.go:71] PutRolePolicy: { PolicyDocument: "{\n \"Statement\": [\n {\n \"Action\": \"ec2:AttachVolume\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"aws:ResourceTag/k8s.io/role/master\": \"1\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": [\n \"s3:Get*\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-read-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/main/*\"\n },\n {\n \"Action\": [\n \"s3:GetObject\",\n \"s3:DeleteObject\",\n \"s3:DeleteObjectVersion\",\n \"s3:PutObject\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:s3:::placeholder-write-bucket/clusters.example.com/nthsqsresources.longclustername.example.com/backups/etcd/events/*\"\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-read-bucket\"\n ]\n },\n {\n \"Action\": [\n \"s3:GetBucketLocation\",\n \"s3:GetEncryptionConfiguration\",\n \"s3:ListBucket\",\n \"s3:ListBucketVersions\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:s3:::placeholder-write-bucket\"\n ]\n },\n {\n \"Action\": [\n \"route53:ChangeResourceRecordSets\",\n \"route53:ListResourceRecordSets\",\n \"route53:GetHostedZone\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::hostedzone/Z1AFAKE1ZON3YO\"\n ]\n },\n {\n \"Action\": [\n \"route53:GetChange\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:route53:::change/*\"\n ]\n },\n {\n \"Action\": [\n \"route53:ListHostedZones\",\n \"route53:ListTagsForResource\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateSecurityGroup\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:security-group/*\"\n ]\n },\n {\n \"Action\": \"ec2:CreateTags\",\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\",\n \"ec2:CreateAction\": [\n \"CreateVolume\",\n \"CreateSnapshot\"\n ]\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"ec2:CreateTags\",\n \"ec2:DeleteTags\"\n ],\n \"Condition\": {\n \"Null\": {\n \"aws:RequestTag/KubernetesCluster\": \"true\"\n },\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": [\n \"arn:aws-test:ec2:*:*:volume/*\",\n \"arn:aws-test:ec2:*:*:snapshot/*\"\n ]\n },\n {\n \"Action\": [\n \"autoscaling:DescribeAutoScalingGroups\",\n \"autoscaling:DescribeAutoScalingInstances\",\n \"autoscaling:DescribeLaunchConfigurations\",\n \"autoscaling:DescribeScalingActivities\",\n \"autoscaling:DescribeTags\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateTags\",\n \"ec2:DeleteRoute\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DescribeAccountAttributes\",\n \"ec2:DescribeAvailabilityZones\",\n \"ec2:DescribeInstanceTypes\",\n \"ec2:DescribeInstances\",\n \"ec2:DescribeLaunchTemplateVersions\",\n \"ec2:DescribeRegions\",\n \"ec2:DescribeRouteTables\",\n \"ec2:DescribeSecurityGroups\",\n \"ec2:DescribeSubnets\",\n \"ec2:DescribeTags\",\n \"ec2:DescribeVolumes\",\n \"ec2:DescribeVolumesModifications\",\n \"ec2:DescribeVpcs\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateTargetGroup\",\n \"elasticloadbalancing:DescribeListeners\",\n \"elasticloadbalancing:DescribeLoadBalancerAttributes\",\n \"elasticloadbalancing:DescribeLoadBalancerPolicies\",\n \"elasticloadbalancing:DescribeLoadBalancers\",\n \"elasticloadbalancing:DescribeTargetGroups\",\n \"elasticloadbalancing:DescribeTargetHealth\",\n \"elasticloadbalancing:RegisterTargets\",\n \"iam:GetServerCertificate\",\n \"iam:ListServerCertificates\",\n \"kms:DescribeKey\",\n \"kms:GenerateRandom\",\n \"sqs:DeleteMessage\",\n \"sqs:ReceiveMessage\"\n ],\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"autoscaling:CompleteLifecycleAction\",\n \"autoscaling:SetDesiredCapacity\",\n \"autoscaling:TerminateInstanceInAutoScalingGroup\",\n \"ec2:AttachVolume\",\n \"ec2:AuthorizeSecurityGroupIngress\",\n \"ec2:DeleteSecurityGroup\",\n \"ec2:DeleteVolume\",\n \"ec2:DetachVolume\",\n \"ec2:ModifyInstanceAttribute\",\n \"ec2:ModifyVolume\",\n \"ec2:RevokeSecurityGroupIngress\",\n \"elasticloadbalancing:AddTags\",\n \"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer\",\n \"elasticloadbalancing:AttachLoadBalancerToSubnets\",\n \"elasticloadbalancing:ConfigureHealthCheck\",\n \"elasticloadbalancing:CreateLoadBalancerListeners\",\n \"elasticloadbalancing:CreateLoadBalancerPolicy\",\n \"elasticloadbalancing:DeleteListener\",\n \"elasticloadbalancing:DeleteLoadBalancer\",\n \"elasticloadbalancing:DeleteLoadBalancerListeners\",\n \"elasticloadbalancing:DeleteTargetGroup\",\n \"elasticloadbalancing:DeregisterInstancesFromLoadBalancer\",\n \"elasticloadbalancing:DeregisterTargets\",\n \"elasticloadbalancing:DetachLoadBalancerFromSubnets\",\n \"elasticloadbalancing:ModifyListener\",\n \"elasticloadbalancing:ModifyLoadBalancerAttributes\",\n \"elasticloadbalancing:ModifyTargetGroup\",\n \"elasticloadbalancing:RegisterInstancesWithLoadBalancer\",\n \"elasticloadbalancing:RegisterTargets\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer\",\n \"elasticloadbalancing:SetLoadBalancerPoliciesOfListener\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:ResourceTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": [\n \"ec2:CreateSecurityGroup\",\n \"ec2:CreateSnapshot\",\n \"ec2:CreateVolume\",\n \"elasticloadbalancing:CreateListener\",\n \"elasticloadbalancing:CreateLoadBalancer\",\n \"elasticloadbalancing:CreateTargetGroup\"\n ],\n \"Condition\": {\n \"StringEquals\": {\n \"aws:RequestTag/KubernetesCluster\": \"nthsqsresources.longclustername.example.com\"\n }\n },\n \"Effect\": \"Allow\",\n \"Resource\": \"*\"\n },\n {\n \"Action\": \"ec2:CreateSecurityGroup\",\n \"Effect\": \"Allow\",\n \"Resource\": \"arn:aws-test:ec2:*:*:vpc/*\"\n }\n ],\n \"Version\": \"2012-10-17\"\n}", PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.716068 46260 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0216 02:06:28.716239 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.716325 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.716366 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.716460 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.716737 46260 routetable.go:149] CreateRoute: { DestinationIpv6CidrBlock: "::/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:28.716870 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-udp-1to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0216 02:06:28.717037 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-1to2379-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.717172 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-4003to65535-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0216 02:06:28.717261 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.717409 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", IpRanges: [{ CidrIp: "0.0.0.0/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-0.0.0.0/0" }] }] } I0216 02:06:28.717494 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.717552 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.717594 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.717726 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.717790 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.717848 46260 routetable.go:149] CreateRoute: { DestinationCidrBlock: "0.0.0.0/0", GatewayId: "igw-2", RouteTableId: "rtb-1" } I0216 02:06:28.717913 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.718069 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-nodes.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" }] }] } I0216 02:06:28.718227 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-22to22-masters.nthsqsresources.longclustername.example.com" }] }] } I0216 02:06:28.718383 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "association.subnet-id", Values: ["subnet-1"] }] } I0216 02:06:28.718434 46260 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-1", SubnetId: "subnet-1" } I0216 02:06:28.718476 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" },{ Key: "Name", Value: "from-masters.nthsqsresources.longclustername.example.com-ingress-all-0to0-masters.nthsqsresources.longclustername.example.com" }] }] } I0216 02:06:28.718630 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-ingress-tcp-2382to4000-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.718787 46260 securitygroups.go:246] AuthorizeSecurityGroupEgress: { GroupId: "sg-1", IpPermissions: [{ IpProtocol: "-1", Ipv6Ranges: [{ CidrIpv6: "::/0" }] }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-nodes.nthsqsresources.longclustername.example.com-egress-all-0to0-::/0" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.718935 46260 securitygroups.go:357] AuthorizeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [{ FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 }], TagSpecifications: [{ ResourceType: "security-group-rule", Tags: [{ Key: "Name", Value: "from-0.0.0.0/0-ingress-tcp-443to443-masters.nthsqsresources.longclustername.example.com" },{ Key: "KubernetesCluster", Value: "nthsqsresources.longclustername.example.com" },{ Key: "kubernetes.io/cluster/nthsqsresources.longclustername.example.com", Value: "owned" }] }] } I0216 02:06:28.719160 46260 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0216 02:06:28.719385 46260 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0216 02:06:28.719537 46260 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0216 02:06:28.719578 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.719611 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.719664 46260 dns.go:238] Pre-creating DNS records I0216 02:06:28.719693 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "CREATE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "CREATE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.727026 46260 zones.go:93] ListHostedZonesPages { } ********************************************************************************* A new kops version is available: 1.23.2 Upgrading is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_kops.md#1.23.2 ********************************************************************************* ********************************************************************************* Kops support for this Kubernetes version is deprecated and will be removed in a future release. Upgrading Kubernetes is recommended More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.22.0 ********************************************************************************* ********************************************************************************* A new kubernetes version is available: 1.20.15 Upgrading is recommended (try kops upgrade cluster) More information: https://github.com/kubernetes/kops/blob/master/permalinks/upgrade_k8s.md#1.20.15 ********************************************************************************* I0216 02:06:28.730038 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.730124 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.801937 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.916930 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.918058 46260 executor.go:111] Tasks: 0 done / 101 total; 45 can run I0216 02:06:28.918323 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.918463 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.918558 46260 zones.go:40] GetHostedZone { Id: "Z1AFAKE1ZON3YO" } I0216 02:06:28.918547 46260 keypairs.go:95] DescribeKeyPairs: { KeyNames: ["kubernetes.nthsqsresources.longclustername.example.com-c4:a6:ed:9a:a8:89:b9:e2:c3:9c:d6:63:eb:9c:71:57"] } I0216 02:06:28.918652 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-events.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.918817 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.918934 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.918969 46260 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.919004 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.etcd-main.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.919161 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.920506 46260 executor.go:111] Tasks: 45 done / 101 total; 24 can run I0216 02:06:28.920618 46260 iamrole.go:161] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0216 02:06:28.920743 46260 iamrole.go:161] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0216 02:06:28.920694 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["us-test-1a.nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.920852 46260 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.920899 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["nodes.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["nodes.nthsqsresources.longclustername.example.com"] } ] } I0216 02:06:28.921067 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } I0216 02:06:28.921119 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.921258 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.921344 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [ { Name: "tag:Name", Values: ["masters.nthsqsresources.longclustername.example.com"] }, { Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }, { Name: "vpc-id", Values: ["vpc-1"] }, { Name: "group-name", Values: ["masters.nthsqsresources.longclustername.example.com"] } ] } I0216 02:06:28.921529 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.921592 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:Name", Values: ["nthsqsresources.longclustername.example.com"] },{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.921710 46260 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.923565 46260 executor.go:111] Tasks: 69 done / 101 total; 26 can run I0216 02:06:28.923646 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.923704 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.923789 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.924012 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.924347 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.924506 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.924691 46260 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: ["k8s-1.4-debian-jessie-amd64-hvm-ebs-2016-10-21"] }], Owners: ["383156758163"] } I0216 02:06:28.926535 46260 executor.go:111] Tasks: 95 done / 101 total; 2 can run I0216 02:06:28.926757 46260 executor.go:111] Tasks: 97 done / 101 total; 4 can run I0216 02:06:28.926899 46260 executor.go:111] Tasks: 101 done / 101 total; 0 can run I0216 02:06:28.929803 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400000"] }] } I0216 02:06:28.929863 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-11400001"] }] } I0216 02:06:28.929900 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-12345678"] }] } I0216 02:06:28.929936 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["ami-15000000"] }] } I0216 02:06:28.929971 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["dopt-1"] }] } I0216 02:06:28.930008 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-12345678"] }] } I0216 02:06:28.930043 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-b2345678"] }] } I0216 02:06:28.930086 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-1"] }] } I0216 02:06:28.930124 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["igw-2"] }] } I0216 02:06:28.930166 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["key-1"] }] } I0216 02:06:28.930206 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-1"] }] } I0216 02:06:28.930252 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["lt-2"] }] } I0216 02:06:28.930299 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-a2345678"] }] } I0216 02:06:28.930338 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["nat-b2345678"] }] } I0216 02:06:28.930373 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-1"] }] } I0216 02:06:28.930412 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["rtb-12345678"] }] } I0216 02:06:28.930451 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.930490 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-2"] }] } I0216 02:06:28.930530 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.930571 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-12345678"] }] } I0216 02:06:28.930609 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-abcdef"] }] } I0216 02:06:28.930661 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-b2345678"] }] } I0216 02:06:28.930700 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-1"] }] } I0216 02:06:28.930742 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vol-2"] }] } I0216 02:06:28.930781 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.930821 46260 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-12345678"] }] } W0216 02:06:28.932327 46260 instances.go:27] MockEc2::DescribeInstances is stub-implemented I0216 02:06:28.932353 46260 keypairs.go:95] DescribeKeyPairs: { } I0216 02:06:28.932398 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932453 46260 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932513 46260 volumes.go:128] DescribeVolumes: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932577 46260 dhcpoptions.go:33] DescribeDhcpOptions: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932630 46260 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932686 46260 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932741 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932790 46260 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932842 46260 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932900 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["nthsqsresources.longclustername.example.com"] }] } I0216 02:06:28.932945 46260 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/nthsqsresources.longclustername.example.com"] }] } W0216 02:06:28.933003 46260 api.go:53] PageSize not implemented I0216 02:06:28.933025 46260 loadbalancers.go:31] DescribeLoadBalancers v2 { PageSize: 20 } W0216 02:06:28.933052 46260 loadbalancers.go:34] PageSize not implemented I0216 02:06:28.933072 46260 targetgroups.go:32] DescribeTargetGroups { PageSize: 20 } W0216 02:06:28.933098 46260 targetgroups.go:35] PageSize not implemented I0216 02:06:28.933116 46260 tags.go:64] DescribeTags v2 { ResourceArns: ["arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-3/3","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-1/1","arn:aws-test:elasticloadbalancing:us-test-1:000000000000:targetgroup/my-external-tg-2/2"] } I0216 02:06:28.933175 46260 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.933207 46260 records.go:48] ListResourceRecordSetsPages { HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } I0216 02:06:28.933256 46260 iaminstanceprofile.go:193] ListInstanceProfiles: { } I0216 02:06:28.933292 46260 iamrole.go:89] ListRoles: { } I0216 02:06:28.933335 46260 internetgateways.go:101] DescribeInternetGateways: { } I0216 02:06:28.933376 46260 routetable.go:53] DescribeRouteTables: { } I0216 02:06:28.933407 46260 routetable.go:53] DescribeRouteTables: { RouteTableIds: ["rtb-1"] } I0216 02:06:28.933883 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.933911 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-1"] } autoscaling-group:nodes.nthsqsresources.longclustername.example.com ok I0216 02:06:28.933948 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "nodes.nthsqsresources.longclustername.example.com" } iam-instance-profile:nodes.nthsqsresources.longclustername.example.com ok autoscaling-group:master-us-test-1a.masters.nthsqsresources.longclustername.example.com ok I0216 02:06:28.933975 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-1", IpPermissions: [{ FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-1" }] },{ IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }] } I0216 02:06:28.934007 46260 iaminstanceprofile.go:156] RemoveRoleFromInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.933881 46260 records.go:89] ChangeResourceRecordSets { ChangeBatch: { Changes: [{ Action: "DELETE", ResourceRecordSet: { Name: "api.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "kops-controller.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } },{ Action: "DELETE", ResourceRecordSet: { Name: "api.internal.nthsqsresources.longclustername.example.com.", ResourceRecords: [{ Value: "203.0.113.123" }], TTL: 10, Type: "A" } }] }, HostedZoneId: "/hostedzone/Z1AFAKE1ZON3YO" } route53-record:Z1AFAKE1ZON3YO/A/api.nthsqsresources.longclustername.example.com. ok I0216 02:06:28.934080 46260 iaminstanceprofile.go:241] DeleteInstanceProfile: { InstanceProfileName: "masters.nthsqsresources.longclustername.example.com" } iam-instance-profile:masters.nthsqsresources.longclustername.example.com ok W0216 02:06:28.934064 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.934125 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-1" } security-group:sg-1 ok I0216 02:06:28.934168 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-2" } volume:vol-2 ok I0216 02:06:28.934230 46260 internetgateways.go:101] DescribeInternetGateways: { InternetGatewayIds: ["igw-2"] } I0216 02:06:28.934291 46260 internetgateways.go:227] DeleteInternetGateway: { InternetGatewayId: "igw-2" } internet-gateway:igw-2 ok I0216 02:06:28.934323 46260 keypairs.go:154] DeleteKeyPair: { KeyPairId: "key-1" } keypair:key-1 ok autoscaling-config:lt-1 ok autoscaling-config:lt-2 ok I0216 02:06:28.934421 46260 securitygroups.go:115] DescribeSecurityGroups: { GroupIds: ["sg-2"] } I0216 02:06:28.934474 46260 securitygroups.go:210] RevokeSecurityGroupIngress: { GroupId: "sg-2", IpPermissions: [ { FromPort: 1, IpProtocol: "udp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 1, IpProtocol: "tcp", ToPort: 2379, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 4003, IpProtocol: "tcp", ToPort: 65535, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 22, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 22 }, { IpProtocol: "-1", UserIdGroupPairs: [{ GroupId: "sg-2" }] }, { FromPort: 2382, IpProtocol: "tcp", ToPort: 4000, UserIdGroupPairs: [{ GroupId: "sg-1" }] }, { FromPort: 443, IpProtocol: "tcp", IpRanges: [{ CidrIp: "0.0.0.0/0" }], ToPort: 443 } ] } W0216 02:06:28.934700 46260 securitygroups.go:228] RevokeSecurityGroupIngress mock not implemented - does not actually revoke permissions I0216 02:06:28.934720 46260 securitygroups.go:79] DeleteSecurityGroup: { GroupId: "sg-2" } security-group:sg-2 ok I0216 02:06:28.934763 46260 volumes.go:203] DeleteVolume: { VolumeId: "vol-1" } volume:vol-1 ok I0216 02:06:28.934847 46260 subnets.go:238] DeleteSubnet: { SubnetId: "subnet-1" } subnet:subnet-1 ok I0216 02:06:28.934867 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.934916 46260 iamrole.go:180] ListAttachedRolePolicies: nodes.nthsqsresources.longclustername.example.com I0216 02:06:28.934928 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "nodes.nthsqsresources.longclustername.example.com", RoleName: "nodes.nthsqsresources.longclustername.example.com" } I0216 02:06:28.934951 46260 iamrole.go:137] DeleteRole: { RoleName: "nodes.nthsqsresources.longclustername.example.com" } iam-role:nodes.nthsqsresources.longclustername.example.com ok I0216 02:06:28.934972 46260 iamrolepolicy.go:108] ListRolePolicies: { RoleName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.934996 46260 iamrole.go:180] ListAttachedRolePolicies: masters.nthsqsresources.longclustername.example.com I0216 02:06:28.935003 46260 iamrolepolicy.go:156] DeleteRolePolicy: { PolicyName: "masters.nthsqsresources.longclustername.example.com", RoleName: "masters.nthsqsresources.longclustername.example.com" } I0216 02:06:28.935022 46260 iamrole.go:137] DeleteRole: { RoleName: "masters.nthsqsresources.longclustername.example.com" } iam-role:masters.nthsqsresources.longclustername.example.com ok I0216 02:06:28.935064 46260 routetable.go:188] DeleteRouteTable: { RouteTableId: "rtb-1" } route-table:rtb-1 ok I0216 02:06:28.935109 46260 vpcs.go:222] DeleteVpc: { VpcId: "vpc-1" } vpc:vpc-1 ok I0216 02:06:28.935153 46260 dhcpoptions.go:173] DeleteDhcpOptions: { DhcpOptionsId: "dopt-1" } dhcp-options:dopt-1 ok I0216 02:06:28.935356 46260 featureflag.go:160] FeatureFlag "SpecOverrideFlag"=false --- PASS: TestLifecycleNodeTerminationHandlerQueueProcessor (0.42s) === RUN TestGetFilters --- PASS: TestGetFilters (0.00s) === RUN TestGetInstanceSelectorOpts --- PASS: TestGetInstanceSelectorOpts (0.00s) === RUN TestValidateAllPrivateOrPublicSubnets --- PASS: TestValidateAllPrivateOrPublicSubnets (0.00s) === RUN TestValidateUserSubnetsWithClusterSubnets --- PASS: TestValidateUserSubnetsWithClusterSubnets (0.00s) === RUN TestCreateInstanceGroup --- PASS: TestCreateInstanceGroup (0.00s) === RUN TestDecorateWithInstanceGroupSpecs --- PASS: TestDecorateWithInstanceGroupSpecs (0.00s) === RUN TestDecorateWithMixedInstancesPolicy --- PASS: TestDecorateWithMixedInstancesPolicy (0.00s) === RUN TestDecorateWithClusterAutoscalerLabels --- PASS: TestDecorateWithClusterAutoscalerLabels (0.00s) === RUN TestNewTemplateContext --- PASS: TestNewTemplateContext (0.00s) PASS ok k8s.io/kops/cmd/kops 52.049s === RUN TestBuildProxyRequest === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/livez === RUN TestBuildProxyRequest/http://127.0.0.1:3990/healthz === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ready === RUN TestBuildProxyRequest/http://127.0.0.1:3990/ === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz? === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose === RUN TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude --- PASS: TestBuildProxyRequest (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/livez (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/healthz (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ready (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/ (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz/foo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyzfoo (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz? (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?foo=bar (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&exclude=2 (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude=1&verbose (0.00s) --- PASS: TestBuildProxyRequest/http://127.0.0.1:3990/readyz?exclude (0.00s) PASS ok k8s.io/kops/cmd/kube-apiserver-healthcheck 0.005s === RUN TestAliasForNodesInRole --- PASS: TestAliasForNodesInRole (0.00s) === RUN TestEnsureDotSuffix --- PASS: TestEnsureDotSuffix (0.00s) === RUN TestParseZoneSpec --- PASS: TestParseZoneSpec (0.00s) === RUN TestParseZoneRules I0216 02:05:40.357073 46704 zonespec.go:75] No rules specified, will permit management of all zones --- PASS: TestParseZoneRules (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/dns 0.004s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) PASS ok k8s.io/kops/dns-controller/pkg/util 0.004s === RUN TestPodController I0216 02:05:40.378393 46706 pod.go:59] starting pod controller I0216 02:05:40.378606 46706 stoppable.go:62] shutting down controller --- PASS: TestPodController (0.00s) PASS I0216 02:05:40.378694 46706 pod.go:107] Got stop signal ok k8s.io/kops/dns-controller/pkg/watchers 0.027s === RUN TestEquivalent --- PASS: TestEquivalent (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider 0.005s Parsing flags. Getting new test interface. Running tests... === RUN TestZonesList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011f200} route53_test.go:93: Got zone list: [0xc00011f230] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc0001260b0} --- PASS: TestZonesList (0.00s) === RUN TestZonesID route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011f360} route53_test.go:93: Got zone list: [0xc00011f390] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc0001260c8} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess route53_test.go:79: Got zones {0xc00011f490} route53_test.go:170: Successfully added managed DNS zone: &{0xc00032b6b0 0xc0001260e8} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011f650} route53_test.go:93: Got zone list: [0xc00011f680] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126108} route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011f7e0} route53_test.go:93: Got zone list: [0xc00011f810] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126130} route53_test.go:187: Successfully added resource record set: {0xc000172600 0xc000126150} --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011faf0} route53_test.go:93: Got zone list: [0xc00011fb20] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc0001261e0} route53_test.go:199: Successfully added resource record set: {0xc000172780 0xc000126200} route53_test.go:118: Got 1 recordsets: [0xc00011fcc0] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc00011fe50} route53_test.go:93: Got zone list: [0xc00011fe80] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc0001262a0} route53_test.go:221: Successfully added resource record set: {0xc000172900 0xc0001262c0} route53_test.go:228: Correctly failed to add duplicate resource record {0xc000172900 0xc0001262c0}: attempt to create duplicate rrset www11.example.com::A --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360240} route53_test.go:93: Got zone list: [0xc000360270] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126380} route53_test.go:246: Successfully removed resource set {0xc000172b00 0xc0001263a0} after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360550} route53_test.go:93: Got zone list: [0xc000360580] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126430} route53_test.go:264: Successfully removed resource set {0xc000172c80 0xc000126450} after adding route53_test.go:118: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360890} route53_test.go:93: Got zone list: [0xc0003608c0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc0001264e8} commontests.go:192: Successfully added resource record set: {0xc000172e00 0xc000126508} commontests.go:45: Correctly replaced resource record {0xc000172e00 0xc000126508} -> {0xc000172f00 0xc000126558} commontests.go:142: Got 1 recordsets: [0xc000360bd0] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000360e00} route53_test.go:93: Got zone list: [0xc000360e30] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126610} commontests.go:192: Successfully added resource record set: {0xc000173180 0xc000126630} commontests.go:70: Correctly replaced resource record {0xc000173180 0xc000126630} -> {0xc000173280 0xc000126680} commontests.go:142: Got 1 recordsets: [0xc000361140] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc0003613c0} route53_test.go:93: Got zone list: [0xc0003613f0] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126748} commontests.go:192: Successfully added resource record set: {0xc000173500 0xc000126768} commontests.go:142: Got 2 recordsets: [0xc000361660 0xc000361680] commontests.go:142: Got 2 recordsets: [0xc0003616f0 0xc000361710] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract route53_test.go:87: Getting zones route53_test.go:79: Got zones {0xc000361910} route53_test.go:93: Got zone list: [0xc000361940] route53_test.go:98: Got at least 1 zone in list:&{0xc00032b470 0xc000126870} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/aws/route53 0.016s === RUN TestZonesList --- PASS: TestZonesList (0.00s) === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestRemove --- PASS: TestRemove (0.00s) === RUN TestNewZone --- PASS: TestNewZone (0.00s) === RUN TestNewResourceRecordSet --- PASS: TestNewResourceRecordSet (0.00s) === RUN TestResourceRecordChangeset --- PASS: TestResourceRecordChangeset (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/do 0.006s === RUN TestZonesList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc0007147c0} clouddns_test.go:71: Got zone list: [0xc000156708] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc0001566a8} --- PASS: TestZonesList (0.00s) === RUN TestZonesID clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc0007148c0} clouddns_test.go:71: Got zone list: [0xc000156888] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000156828} --- PASS: TestZonesID (0.00s) === RUN TestZoneAddSuccess clouddns_test.go:131: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc0007149c0} clouddns_test.go:133: Got zones, making new Zone clouddns_test.go:149: Successfully added managed DNS zone: &{0xc00014df80 0xc000156a08} --- PASS: TestZoneAddSuccess (0.00s) === RUN TestResourceRecordSetsList clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000714a80} clouddns_test.go:71: Got zone list: [0xc000156b88] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000156b28} clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsList (0.00s) === RUN TestResourceRecordSetsAddSuccess clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000714ba0} clouddns_test.go:71: Got zone list: [0xc000156d80] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000156d20} clouddns_test.go:165: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> --- PASS: TestResourceRecordSetsAddSuccess (0.00s) === RUN TestResourceRecordSetsAdditionVisible clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000714ce0} clouddns_test.go:71: Got zone list: [0xc0001570c8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000157098} clouddns_test.go:176: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:96: Got 1 recordsets: [<(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>] --- PASS: TestResourceRecordSetsAdditionVisible (0.00s) === RUN TestResourceRecordSetsAddDuplicateFailure clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000714e40} clouddns_test.go:71: Got zone list: [0xc000157350] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc0001572a8} clouddns_test.go:198: Successfully added resource record set: <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> clouddns_test.go:205: Correctly failed to add duplicate resource record <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180>: attempt to insert duplicate rrset {www11.example.com [10.10.10.10 169.20.20.20] 180 A} --- PASS: TestResourceRecordSetsAddDuplicateFailure (0.00s) === RUN TestResourceRecordSetsRemove clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000714fa0} clouddns_test.go:71: Got zone list: [0xc0001574e8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc0001574b8} clouddns_test.go:223: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding --- PASS: TestResourceRecordSetsRemove (0.00s) === RUN TestResourceRecordSetsRemoveGone clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc0007150e0} clouddns_test.go:71: Got zone list: [0xc000157650] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000157620} clouddns_test.go:241: Successfully removed resource set <(clouddns) "www11.example.com" type=A rrdatas=["10.10.10.10" "169.20.20.20"] ttl=180> after adding clouddns_test.go:96: Got 0 recordsets: [] --- PASS: TestResourceRecordSetsRemoveGone (0.00s) === RUN TestResourceRecordSetsReplace clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000715240} clouddns_test.go:71: Got zone list: [0xc000157818] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc0001577e8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:45: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] --- PASS: TestResourceRecordSetsReplace (0.00s) === RUN TestResourceRecordSetsReplaceAll clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc0007153a0} clouddns_test.go:71: Got zone list: [0xc000157ae8] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000157ab8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:70: Correctly replaced resource record <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> -> <(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80> commontests.go:142: Got 1 recordsets: [<(clouddns) "beta.test.com" type=A rrdatas=["8.8.8.8"] ttl=80>] commontests.go:117: Did not Get recordset: alpha.test.com --- PASS: TestResourceRecordSetsReplaceAll (0.00s) === RUN TestResourceRecordSetsDifferentTypes clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000715520} clouddns_test.go:71: Got zone list: [0xc000157e18] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc000157de8} commontests.go:192: Successfully added resource record set: <(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] commontests.go:142: Got 2 recordsets: [<(clouddns) "alpha.test.com" type=A rrdatas=["8.8.4.4"] ttl=40> <(clouddns) "alpha.test.com" type=AAAA rrdatas=["2001:4860:4860::8888"] ttl=80>] --- PASS: TestResourceRecordSetsDifferentTypes (0.00s) === RUN TestContract clouddns_test.go:66: Getting zones clouddns_test.go:58: Got zones {0xc000149498 0xc000715700} clouddns_test.go:71: Got zone list: [0xc000010210] clouddns_test.go:76: Got at least 1 zone in list:&{0xc00014de00 0xc0000101e0} --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/google/clouddns 0.060s === RUN TestContract --- PASS: TestContract (0.00s) PASS ok k8s.io/kops/dnsprovider/pkg/dnsprovider/providers/openstack/designate 0.022s === RUN TestBootstarapBuilder_Simple --- PASS: TestBootstarapBuilder_Simple (0.00s) PASS ok k8s.io/kops/nodeup/pkg/bootstrap 0.076s === RUN TestBuildAzure --- PASS: TestBuildAzure (0.00s) === RUN TestBuildAWSCustomNodeIPFamilies --- PASS: TestBuildAWSCustomNodeIPFamilies (0.00s) === RUN TestContainerdBuilder_Docker_19_03_13 I0216 02:05:48.892231 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.892456 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.892493 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.892531 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.892578 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.892613 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.892630 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.892661 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.892689 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.903052 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_13 (0.02s) === RUN TestContainerdBuilder_Docker_19_03_14 I0216 02:05:48.908996 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.909270 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.909321 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.909368 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.909421 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.909487 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.909519 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.909558 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.909585 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.911465 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdBuilder_Docker_19_03_14 (0.01s) === RUN TestContainerdBuilder_Simple I0216 02:05:48.918856 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.919083 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.919145 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.919189 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.919235 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.919278 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.919311 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.919349 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.919375 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.921003 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.923285 47290 assetstore.go:108] Matching assets for "^bin/(containerd|ctr)": I0216 02:05:48.923311 47290 assetstore.go:111] bin/containerd I0216 02:05:48.923328 47290 assetstore.go:111] bin/containerd-shim I0216 02:05:48.923341 47290 assetstore.go:111] bin/containerd-shim-runc-v1 I0216 02:05:48.923355 47290 assetstore.go:111] bin/containerd-shim-runc-v2 I0216 02:05:48.923369 47290 assetstore.go:111] bin/containerd-stress I0216 02:05:48.923383 47290 assetstore.go:111] bin/ctr I0216 02:05:48.923444 47290 assetstore.go:108] Matching assets for "/runc\\.(amd64|arm64)$": I0216 02:05:48.923466 47290 assetstore.go:111] https://github.com/opencontainers/runc/releases/download/v1.1.0/runc.amd64 --- PASS: TestContainerdBuilder_Simple (0.01s) === RUN TestContainerdBuilder_Flatcar I0216 02:05:48.928213 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.928415 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.928455 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.928493 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.928537 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.928573 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.928604 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.928637 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.928663 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.930204 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.932324 47290 containerd.go:58] Detected Flatcar; won't install containerd --- PASS: TestContainerdBuilder_Flatcar (0.01s) === RUN TestContainerdBuilder_SkipInstall I0216 02:05:48.933659 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.933865 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.933903 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.933938 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.933992 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.934107 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.934137 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.934172 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.934195 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.935636 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.937947 47290 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestContainerdBuilder_SkipInstall (0.00s) === RUN TestContainerdBuilder_BuildFlags --- PASS: TestContainerdBuilder_BuildFlags (0.00s) === RUN TestContainerdConfig --- PASS: TestContainerdConfig (0.00s) === RUN TestAppendGPURuntimeContainerdConfig --- PASS: TestAppendGPURuntimeContainerdConfig (0.00s) === RUN TestDockerBuilder_19_03_11 I0216 02:05:48.938894 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.939097 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.939133 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.939175 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.939216 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.939286 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.939318 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.939354 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.939378 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.940899 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.944128 47290 assetstore.go:108] Matching assets for "^docker/": I0216 02:05:48.944155 47290 assetstore.go:111] docker/containerd I0216 02:05:48.944164 47290 assetstore.go:111] docker/containerd-shim I0216 02:05:48.944171 47290 assetstore.go:111] docker/ctr I0216 02:05:48.944177 47290 assetstore.go:111] docker/docker I0216 02:05:48.944184 47290 assetstore.go:111] docker/docker-init I0216 02:05:48.944191 47290 assetstore.go:111] docker/docker-proxy I0216 02:05:48.944198 47290 assetstore.go:111] docker/dockerd I0216 02:05:48.944205 47290 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_19_03_11 (0.01s) === RUN TestDockerBuilder_LogFlags I0216 02:05:48.947476 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.947663 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.947699 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.947728 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.947759 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.947814 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.947834 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.947857 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.947875 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.949069 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.950812 47290 assetstore.go:108] Matching assets for "^docker/": I0216 02:05:48.950845 47290 assetstore.go:111] docker/containerd I0216 02:05:48.950854 47290 assetstore.go:111] docker/containerd-shim I0216 02:05:48.950861 47290 assetstore.go:111] docker/ctr I0216 02:05:48.950869 47290 assetstore.go:111] docker/docker I0216 02:05:48.950876 47290 assetstore.go:111] docker/docker-init I0216 02:05:48.950883 47290 assetstore.go:111] docker/docker-proxy I0216 02:05:48.950890 47290 assetstore.go:111] docker/dockerd I0216 02:05:48.950903 47290 assetstore.go:111] docker/runc --- PASS: TestDockerBuilder_LogFlags (0.01s) === RUN TestDockerBuilder_SkipInstall I0216 02:05:48.953753 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.953919 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.953947 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.953983 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.954010 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.954052 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.954069 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.954087 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.954100 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.955019 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.956551 47290 docker.go:63] SkipInstall is set to true; won't install Docker --- PASS: TestDockerBuilder_SkipInstall (0.00s) === RUN TestDockerBuilder_HealthCheck I0216 02:05:48.956723 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.956853 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.956886 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.956912 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.956947 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.956992 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.957018 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.957041 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.957056 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.958016 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:48.959376 47290 assetstore.go:108] Matching assets for "^docker/": I0216 02:05:48.959396 47290 assetstore.go:111] docker/docker I0216 02:05:48.959410 47290 assetstore.go:111] docker/docker-containerd I0216 02:05:48.959420 47290 assetstore.go:111] docker/docker-containerd-ctr I0216 02:05:48.959427 47290 assetstore.go:111] docker/docker-containerd-shim I0216 02:05:48.959435 47290 assetstore.go:111] docker/docker-init I0216 02:05:48.959441 47290 assetstore.go:111] docker/docker-proxy I0216 02:05:48.959447 47290 assetstore.go:111] docker/docker-runc I0216 02:05:48.959454 47290 assetstore.go:111] docker/dockerd --- PASS: TestDockerBuilder_HealthCheck (0.01s) === RUN TestDockerBuilder_BuildFlags --- PASS: TestDockerBuilder_BuildFlags (0.00s) === RUN TestContainerdHooksBuilder I0216 02:05:48.964561 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.964703 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.964733 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.964757 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.964781 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.964803 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.964820 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.964842 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.964855 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.967152 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestContainerdHooksBuilder (0.00s) === RUN TestDockerHooksBuilder I0216 02:05:48.969364 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.969495 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.969521 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.969546 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.969572 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.969594 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.969611 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.969633 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.969647 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.971333 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestDockerHooksBuilder (0.00s) === RUN TestKopsControllerBuilder I0216 02:05:48.973587 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.973714 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.973739 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.973767 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.973793 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.973814 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.973831 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.973852 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.973866 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.975417 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKopsControllerBuilder (0.01s) === RUN Test_KubeAPIServer_BuildFlags --- PASS: Test_KubeAPIServer_BuildFlags (0.01s) === RUN TestKubeAPIServerBuilder I0216 02:05:48.986345 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.986580 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.986625 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.986662 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.986701 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.986745 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.986766 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.986790 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.986808 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.988785 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilder (0.01s) === RUN TestDeddicatedAPIServerBuilder I0216 02:05:48.997504 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:48.997693 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:48.997738 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:48.997770 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:48.997802 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:48.997839 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:48.997864 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:48.997891 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:48.997927 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:48.999888 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestDeddicatedAPIServerBuilder (0.01s) === RUN TestWithoutEtcdEventsAPIServerBuilder I0216 02:05:49.006807 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:49.006980 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:49.007019 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:49.007047 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:49.007076 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:49.007103 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:49.007120 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:49.007142 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:49.007159 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:49.008977 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestWithoutEtcdEventsAPIServerBuilder (0.01s) === RUN TestAwsIamAuthenticator I0216 02:05:49.017065 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:49.017229 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:49.017264 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:49.017304 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:49.017339 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:49.017365 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:49.017381 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:49.017404 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:49.017420 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:49.019306 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestAwsIamAuthenticator (0.01s) === RUN TestKubeAPIServerBuilderAMD64 I0216 02:05:49.027158 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:49.027289 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:49.027329 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:49.027354 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:49.027380 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:49.027401 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:49.027417 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:49.027437 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:49.027452 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:49.029112 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderAMD64 (1.42s) === RUN TestKubeAPIServerBuilderARM64 I0216 02:05:50.451614 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.451847 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.451898 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.451951 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.452002 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.452042 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.452073 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.452105 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.452132 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.454898 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeAPIServerBuilderARM64 (0.13s) === RUN TestKubeControllerManagerBuilder I0216 02:05:50.584022 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.584194 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.584232 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.584288 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.584333 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.584365 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.584381 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.584404 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.584421 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.586343 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilder (0.02s) === RUN TestKubeControllerManagerBuilderAMD64 I0216 02:05:50.601536 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.601733 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.601773 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.601803 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.601839 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.601866 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.601883 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.601906 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.601922 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.604043 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderAMD64 (0.13s) === RUN TestKubeControllerManagerBuilderARM64 I0216 02:05:50.735828 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.736074 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.736123 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.736177 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.736229 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.736274 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.736308 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.736349 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.736377 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.739729 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeControllerManagerBuilderARM64 (0.13s) === RUN TestKubeProxyBuilder_buildPod === RUN TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 --- PASS: TestKubeProxyBuilder_buildPod (0.00s) --- PASS: TestKubeProxyBuilder_buildPod/Setup_KubeProxy_for_kubernetes_version_1.20 (0.00s) === RUN TestKubeProxyBuilder I0216 02:05:50.867996 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.868192 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.868228 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.868258 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.868298 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.868325 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.868341 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.868365 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.868381 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.870402 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilder (0.01s) === RUN TestKubeProxyBuilderAMD64 I0216 02:05:50.874419 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:50.874667 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:50.874720 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:50.874765 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:50.874817 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:50.874859 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:50.874891 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:50.874928 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:50.874959 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:50.877712 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderAMD64 (0.13s) === RUN TestKubeProxyBuilderARM64 I0216 02:05:51.004200 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.004423 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.004471 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.004524 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.004572 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.004609 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.004639 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.004679 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.004703 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.007587 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestKubeProxyBuilderARM64 (0.13s) === RUN TestParseDefault --- PASS: TestParseDefault (0.00s) === RUN TestParse --- PASS: TestParse (0.00s) === RUN TestKubeSchedulerBuilder I0216 02:05:51.134135 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.134361 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.134410 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.134450 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.134494 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.134532 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.134563 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.134598 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.134623 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.137274 47290 zones.go:93] ListHostedZonesPages { } W0216 02:05:51.139840 47290 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilder (0.01s) === RUN TestKubeSchedulerBuilderAMD64 I0216 02:05:51.141625 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.141789 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.141821 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.141847 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.141875 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.141902 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.141919 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.141941 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.141957 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.143764 47290 zones.go:93] ListHostedZonesPages { } W0216 02:05:51.269339 47290 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderAMD64 (0.13s) === RUN TestKubeSchedulerBuilderARM64 I0216 02:05:51.271887 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.272101 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.272147 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.272192 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.272237 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.272275 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.272305 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.272344 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.272372 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.275000 47290 zones.go:93] ListHostedZonesPages { } W0216 02:05:51.402587 47290 kube_scheduler.go:115] using embedded kubescheduler configuration --- PASS: TestKubeSchedulerBuilderARM64 (0.13s) === RUN TestKubectlBuilder I0216 02:05:51.405826 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.406070 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.406125 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.406180 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.406237 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.406287 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.406327 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.406372 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.406405 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.409713 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:51.413172 47290 assetstore.go:159] Resolved asset kubectl: to /path/to/kubectl/asset --- PASS: TestKubectlBuilder (0.01s) === RUN Test_InstanceGroupKubeletMerge --- PASS: Test_InstanceGroupKubeletMerge (0.00s) === RUN TestTaintsApplied --- PASS: TestTaintsApplied (0.00s) === RUN Test_RunKubeletBuilder I0216 02:05:51.416570 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.416798 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.416873 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.416928 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.416983 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.417035 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.417075 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.417123 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.417155 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.419578 47290 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilder (0.01s) === RUN Test_RunKubeletBuilderWarmPool I0216 02:05:51.425933 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.426205 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.426315 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.426381 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.426447 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.426494 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.426533 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.426578 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.426611 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.428993 47290 zones.go:93] ListHostedZonesPages { } --- PASS: Test_RunKubeletBuilderWarmPool (0.01s) === RUN Test_BuildComponentConfigFile --- PASS: Test_BuildComponentConfigFile (0.00s) === RUN TestProtokubeBuilder I0216 02:05:51.435165 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.435455 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.435535 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.435611 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.435694 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.435767 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.435832 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.435901 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.435959 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.438932 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:51.444551 47290 assetstore.go:108] Matching assets for "protokube$": I0216 02:05:51.444587 47290 assetstore.go:111] /opt/kops/bin/protokube I0216 02:05:51.444599 47290 assetstore.go:129] Found single matching asset for expr "protokube$": "protokube" I0216 02:05:51.444623 47290 assetstore.go:108] Matching assets for "channels$": I0216 02:05:51.444632 47290 assetstore.go:111] /opt/kops/bin/channels I0216 02:05:51.444639 47290 assetstore.go:129] Found single matching asset for expr "channels$": "channels" --- PASS: TestProtokubeBuilder (0.01s) === RUN TestSecretBuilder I0216 02:05:51.446178 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.446372 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.446408 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.446440 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.446472 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.446501 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.446519 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.446542 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.446559 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.448503 47290 zones.go:93] ListHostedZonesPages { } --- PASS: TestSecretBuilder (0.00s) === RUN TestUpdateServiceBuilderAutomaticUpgrade I0216 02:05:51.450997 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.451181 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.451214 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.451243 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.451273 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.451298 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.451315 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.451338 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.451355 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.453077 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:51.454794 47290 update_service.go:96] Detected OS {deb ubuntu focal 20.04}; installing unattended-upgrades package --- PASS: TestUpdateServiceBuilderAutomaticUpgrade (0.00s) === RUN TestUpdateServiceBuilderExternal I0216 02:05:51.455384 47290 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:05:51.455563 47290 internetgateways.go:66] CreateInternetGateway: { } I0216 02:05:51.455597 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:05:51.455627 47290 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:05:51.455660 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:05:51.455687 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:05:51.455705 47290 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:05:51.455732 47290 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:05:51.455749 47290 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:05:51.457539 47290 zones.go:93] ListHostedZonesPages { } I0216 02:05:51.459272 47290 update_service.go:91] UpdatePolicy requests automatic updates; skipping installation of package "unattended-upgrades" --- PASS: TestUpdateServiceBuilderExternal (0.00s) PASS ok k8s.io/kops/nodeup/pkg/model 2.641s === RUN Test_Strategy --- PASS: Test_Strategy (0.00s) === RUN Test_In_StateStore --- PASS: Test_In_StateStore (0.00s) PASS ok k8s.io/kops/pkg/acls/s3 0.033s === RUN TestWarmPoolSpec_IsEnabled === RUN TestWarmPoolSpec_IsEnabled/nil === RUN TestWarmPoolSpec_IsEnabled/empty === RUN TestWarmPoolSpec_IsEnabled/1 === RUN TestWarmPoolSpec_IsEnabled/0 --- PASS: TestWarmPoolSpec_IsEnabled (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/nil (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/empty (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/1 (0.00s) --- PASS: TestWarmPoolSpec_IsEnabled/0 (0.00s) === RUN TestWarmPoolSpec_ResolveDefaults === RUN TestWarmPoolSpec_ResolveDefaults/MinSize === RUN TestWarmPoolSpec_ResolveDefaults/MaxSize === RUN TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook --- PASS: TestWarmPoolSpec_ResolveDefaults (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MinSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/MaxSize (0.00s) --- PASS: TestWarmPoolSpec_ResolveDefaults/EnableLifecycleHook (0.00s) === RUN Test_ParseInstanceGroupRole --- PASS: Test_ParseInstanceGroupRole (0.00s) === RUN TestParseConfigYAML === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" === RUN TestParseConfigYAML/"kubeAPIServer:_{}" --- PASS: TestParseConfigYAML (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3140m_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.14_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{__auditWebhookBatchThrottleQps:_3.140_}" (0.00s) --- PASS: TestParseConfigYAML/"kubeAPIServer:_{}" (0.00s) === RUN TestWeaveParseConfigYAML === RUN TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" === RUN TestWeaveParseConfigYAML/"networking:_{}" --- PASS: TestWeaveParseConfigYAML (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{__weave:_{_memoryRequest:_500Mi,_cpuRequest:_100m,_npcMemoryRequest:_100Mi,_npcCPURequest:_50m}_}" (0.00s) --- PASS: TestWeaveParseConfigYAML/"networking:_{}" (0.00s) === RUN Test_SemverOrdering --- PASS: Test_SemverOrdering (0.00s) PASS ok k8s.io/kops/pkg/apis/kops 0.029s === RUN TestRoundTripTypes --- PASS: TestRoundTripTypes (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/install 0.027s === RUN TestUseCiliumEtcd --- PASS: TestUseCiliumEtcd (0.00s) === RUN Test_FindSubnet --- PASS: Test_FindSubnet (0.00s) === RUN Test_FindZonesForInstanceGroup --- PASS: Test_FindZonesForInstanceGroup (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/model 0.026s === RUN TestGetNodeRole === RUN TestGetNodeRole/RoleNone === RUN TestGetNodeRole/RoleNewerLabel === RUN TestGetNodeRole/RoleOlderLabel --- PASS: TestGetNodeRole (0.00s) --- PASS: TestGetNodeRole/RoleNone (0.00s) --- PASS: TestGetNodeRole/RoleNewerLabel (0.00s) --- PASS: TestGetNodeRole/RoleOlderLabel (0.00s) === RUN Test_ParseKubernetesVersion === RUN Test_ParseKubernetesVersion/1.3.7 === RUN Test_ParseKubernetesVersion/v1.4.0-beta.8 === RUN Test_ParseKubernetesVersion/1.5.0 === RUN Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ === RUN Test_ParseKubernetesVersion/#00 E0216 02:05:48.836665 47292 versions.go:96] unable to parse Kubernetes version "" === RUN Test_ParseKubernetesVersion/abc E0216 02:05:48.836763 47292 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: Test_ParseKubernetesVersion (0.00s) --- PASS: Test_ParseKubernetesVersion/1.3.7 (0.00s) --- PASS: Test_ParseKubernetesVersion/v1.4.0-beta.8 (0.00s) --- PASS: Test_ParseKubernetesVersion/1.5.0 (0.00s) --- PASS: Test_ParseKubernetesVersion/https://storage.googleapis.com/k8s-release-dev/ci/v1.4.0-alpha.2.677+ea69570f61af8e/ (0.00s) --- PASS: Test_ParseKubernetesVersion/#00 (0.00s) --- PASS: Test_ParseKubernetesVersion/abc (0.00s) === RUN Test_IsKubernetesGTEWithPatch --- PASS: Test_IsKubernetesGTEWithPatch (0.00s) === RUN Test_IsKubernetesGTEWithoutPatch --- PASS: Test_IsKubernetesGTEWithoutPatch (0.00s) === RUN Test_IsKubernetesGTEWithPre --- PASS: Test_IsKubernetesGTEWithPre (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/util 0.008s === RUN TestAWSValidateExternalCloudConfig --- PASS: TestAWSValidateExternalCloudConfig (0.00s) === RUN TestValidateInstanceGroupSpec I0216 02:05:49.464343 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464442 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464476 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464503 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464527 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestValidateInstanceGroupSpec (0.00s) === RUN TestMixedInstancePolicies I0216 02:05:49.464733 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464784 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464813 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464834 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464858 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464883 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464921 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464941 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464975 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.464999 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465017 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465047 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465084 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465107 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465125 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465153 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465173 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465192 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } I0216 02:05:49.465210 47655 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: TestMixedInstancePolicies (0.00s) === RUN TestInstanceMetadataOptions I0216 02:05:49.465396 47655 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } I0216 02:05:49.465483 47655 images.go:53] DescribeImages: { Filters: [{ Name: "name", Values: [""] }], Owners: ["self"] } --- PASS: TestInstanceMetadataOptions (0.00s) === RUN TestLoadBalancerSubnets --- PASS: TestLoadBalancerSubnets (0.00s) === RUN TestAWSAuthentication --- PASS: TestAWSAuthentication (0.00s) === RUN TestAWSAdditionalRoutes --- PASS: TestAWSAdditionalRoutes (0.00s) === RUN TestValidEtcdChanges --- PASS: TestValidEtcdChanges (0.00s) === RUN TestValidateInstanceProfile --- PASS: TestValidateInstanceProfile (0.00s) === RUN TestValidMasterInstanceGroup --- PASS: TestValidMasterInstanceGroup (0.00s) === RUN TestValidBootDevice --- PASS: TestValidBootDevice (0.00s) === RUN TestValidNodeLabels --- PASS: TestValidNodeLabels (0.00s) === RUN TestValidateIGCloudLabels --- PASS: TestValidateIGCloudLabels (0.00s) === RUN TestIGCloudLabelIsIGName --- PASS: TestIGCloudLabelIsIGName (0.00s) === RUN TestValidTaints --- PASS: TestValidTaints (0.00s) === RUN TestIGUpdatePolicy === RUN TestIGUpdatePolicy/missing === RUN TestIGUpdatePolicy/automatic === RUN TestIGUpdatePolicy/external === RUN TestIGUpdatePolicy/empty === RUN TestIGUpdatePolicy/unknown --- PASS: TestIGUpdatePolicy (0.00s) --- PASS: TestIGUpdatePolicy/missing (0.00s) --- PASS: TestIGUpdatePolicy/automatic (0.00s) --- PASS: TestIGUpdatePolicy/external (0.00s) --- PASS: TestIGUpdatePolicy/empty (0.00s) --- PASS: TestIGUpdatePolicy/unknown (0.00s) === RUN TestValidInstanceGroup --- PASS: TestValidInstanceGroup (0.00s) === RUN Test_ValidateTopology --- PASS: Test_ValidateTopology (0.00s) === RUN Test_Validate_DNS --- PASS: Test_Validate_DNS (0.00s) === RUN TestValidateCIDR --- PASS: TestValidateCIDR (0.00s) === RUN TestValidateSubnets --- PASS: TestValidateSubnets (0.00s) === RUN TestValidateKubeAPIServer --- PASS: TestValidateKubeAPIServer (0.00s) === RUN Test_Validate_DockerConfig_Storage --- PASS: Test_Validate_DockerConfig_Storage (0.00s) === RUN Test_Validate_Networking_Flannel --- PASS: Test_Validate_Networking_Flannel (0.00s) === RUN Test_Validate_AdditionalPolicies --- PASS: Test_Validate_AdditionalPolicies (0.00s) === RUN Test_Validate_Calico === RUN Test_Validate_Calico/empty_specs === RUN Test_Validate_Calico/positive_Typha_replica_count === RUN Test_Validate_Calico/negative_Typha_replica_count === RUN Test_Validate_Calico/with_etcd_version === RUN Test_Validate_Calico/IPv4_autodetection_method === RUN Test_Validate_Calico/IPv6_autodetection_method === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method === RUN Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter === RUN Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) === RUN Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name === RUN Test_Validate_Calico/AWS_source/destination_checks_off === RUN Test_Validate_Calico/AWS_source/destination_checks_enabled === RUN Test_Validate_Calico/AWS_source/destination_checks_disabled === RUN Test_Validate_Calico/AWS_source/destination_checks_left_as_is === RUN Test_Validate_Calico/encapsulation_none_with_IPv4 === RUN Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 === RUN Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 === RUN Test_Validate_Calico/unknown_Calico_IPIP_mode === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) === RUN Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) === RUN Test_Validate_Calico/Calico_IPv6_without_encapsulation --- PASS: Test_Validate_Calico (0.00s) --- PASS: Test_Validate_Calico/empty_specs (0.00s) --- PASS: Test_Validate_Calico/positive_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/negative_Typha_replica_count (0.00s) --- PASS: Test_Validate_Calico/with_etcd_version (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv6_autodetection_method_missing_parameter (0.00s) --- PASS: Test_Validate_Calico/IPv4_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/IPv6_autodetection_method_with_parameter_list (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(parenthesis) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_(equals) (0.00s) --- PASS: Test_Validate_Calico/invalid_IPv4_autodetection_method_parameter_with_no_name (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_off (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_enabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_disabled (0.00s) --- PASS: Test_Validate_Calico/AWS_source/destination_checks_left_as_is (0.00s) --- PASS: Test_Validate_Calico/encapsulation_none_with_IPv4 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_IPIP_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/encapsulation_mode_VXLAN_for_IPv6 (0.00s) --- PASS: Test_Validate_Calico/unknown_Calico_IPIP_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(implicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPIP_encapsulation_mode_(explicit)_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(always) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(cross-subnet) (0.00s) --- PASS: Test_Validate_Calico/Calico_VXLAN_encapsulation_mode_with_IPIP_IPPool_mode_(never) (0.00s) --- PASS: Test_Validate_Calico/Calico_IPv6_without_encapsulation (0.00s) === RUN Test_Validate_Cilium --- PASS: Test_Validate_Cilium (0.00s) === RUN Test_Validate_RollingUpdate --- PASS: Test_Validate_RollingUpdate (0.00s) === RUN Test_Validate_NodeLocalDNS --- PASS: Test_Validate_NodeLocalDNS (0.00s) === RUN Test_Validate_CloudConfiguration === RUN Test_Validate_CloudConfiguration/neither === RUN Test_Validate_CloudConfiguration/all_false === RUN Test_Validate_CloudConfiguration/all_true === RUN Test_Validate_CloudConfiguration/os_false === RUN Test_Validate_CloudConfiguration/os_true === RUN Test_Validate_CloudConfiguration/all_false,_os_false === RUN Test_Validate_CloudConfiguration/all_false,_os_true === RUN Test_Validate_CloudConfiguration/all_true,_os_false === RUN Test_Validate_CloudConfiguration/all_true,_os_true --- PASS: Test_Validate_CloudConfiguration (0.00s) --- PASS: Test_Validate_CloudConfiguration/neither (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_false,_os_true (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_false (0.00s) --- PASS: Test_Validate_CloudConfiguration/all_true,_os_true (0.00s) === RUN TestValidateSAExternalPermissions === RUN TestValidateSAExternalPermissions/Duplicate_SA === RUN TestValidateSAExternalPermissions/Missing_permissions === RUN TestValidateSAExternalPermissions/Setting_both_arn_and_inline === RUN TestValidateSAExternalPermissions/Empty_SA_name === RUN TestValidateSAExternalPermissions/Empty_SA_namespace --- PASS: TestValidateSAExternalPermissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Duplicate_SA (0.00s) --- PASS: TestValidateSAExternalPermissions/Missing_permissions (0.00s) --- PASS: TestValidateSAExternalPermissions/Setting_both_arn_and_inline (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_name (0.00s) --- PASS: TestValidateSAExternalPermissions/Empty_SA_namespace (0.00s) === RUN Test_Validate_Nvidia_Cluster --- PASS: Test_Validate_Nvidia_Cluster (0.00s) === RUN Test_Validate_Nvidia_Ig --- PASS: Test_Validate_Nvidia_Ig (0.00s) PASS ok k8s.io/kops/pkg/apis/kops/validation 0.105s === RUN TestApplySet I0216 02:05:49.400163 47656 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0216 02:05:49.400918 47656 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0216 02:05:49.401953 47656 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0216 02:05:49.402519 47656 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0216 02:05:49.404060 47656 apiserver.go:85] kubeapiserver request: GET /api?timeout=32s I0216 02:05:49.404624 47656 apiserver.go:85] kubeapiserver request: GET /apis?timeout=32s I0216 02:05:49.405200 47656 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0216 02:05:49.405258 47656 apiserver.go:85] kubeapiserver request: GET /api/v1?timeout=32s I0216 02:05:49.406033 47656 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset?fieldManager=kops&force=true I0216 02:05:49.406119 47656 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"name\":\"test-applyset\"}}" I0216 02:05:49.406474 47656 health.go:33] status conditions not found for /v1, Kind=Namespace I0216 02:05:49.407189 47656 apiserver.go:85] kubeapiserver request: PATCH /api/v1/namespaces/test-applyset/configmaps/foo?fieldManager=kops&force=true I0216 02:05:49.407312 47656 patchresource.go:57] patch request "{\"apiVersion\":\"v1\",\"data\":{\"foo\":\"bar\"},\"kind\":\"ConfigMap\",\"metadata\":{\"name\":\"foo\",\"namespace\":\"test-applyset\"}}" I0216 02:05:49.407657 47656 health.go:33] status conditions not found for /v1, Kind=ConfigMap I0216 02:05:49.407926 47656 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset I0216 02:05:49.407971 47656 getresource.go:64] returning {"apiVersion":"v1","kind":"Namespace","metadata":{"labels":{"kubernetes.io/metadata.name":"test-applyset"},"name":"test-applyset"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} I0216 02:05:49.408656 47656 apiserver.go:85] kubeapiserver request: GET /api/v1/namespaces/test-applyset/configmaps/foo I0216 02:05:49.408692 47656 getresource.go:64] returning {"apiVersion":"v1","data":{"foo":"bar"},"kind":"ConfigMap","metadata":{"name":"foo","namespace":"test-applyset"}} --- PASS: TestApplySet (0.01s) PASS ok k8s.io/kops/pkg/applylib/applyset 0.039s === RUN TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub W0216 02:05:49.498455 47659 builder.go:231] failed to digest image "proxy.example.com/weaveworks/weave-kube" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToDockerHub (0.10s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub W0216 02:05:49.499680 47659 builder.go:231] failed to digest image "proxy.example.com/debian" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedDockerHub (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL W0216 02:05:49.500580 47659 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToSimplifiedKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL W0216 02:05:49.501839 47659 builder.go:231] failed to digest image "proxy.example.com/google_containers/kube-apiserver" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToLegacyKubernetesURL (0.00s) === RUN TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags W0216 02:05:49.502776 47659 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerProxy_AppliesToImagesWithTags (0.00s) === RUN TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges W0216 02:05:49.503706 47659 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" W0216 02:05:49.504521 47659 builder.go:231] failed to digest image "proxy.example.com/kube-apiserver:1.2.3" --- PASS: TestValidate_RemapImage_ContainerRegistry_MappingMultipleTimesConverges (0.00s) === RUN TestRemapEmptySection --- PASS: TestRemapEmptySection (0.00s) === RUN Test_BuildVFSPath E0216 02:05:49.505334 47659 copyfile.go:205] Unable to determine VFS path from supplied URL: https://foo/k8s-for-greeks-kops/kubernetes-release/release/v1.7.2/bin/linux/amd64/kubectl E0216 02:05:49.505354 47659 copyfile.go:206] S3, Google Cloud Storage, and File Paths are supported. E0216 02:05:49.505365 47659 copyfile.go:207] For S3, please make sure that the supplied file repository URL adhere to S3 naming conventions, https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region. E0216 02:05:49.505375 47659 copyfile.go:208] For GCS, please make sure that the supplied file repository URL adheres to https://storage.googleapis.com/ --- PASS: Test_BuildVFSPath (0.00s) PASS ok k8s.io/kops/pkg/assets 0.135s === RUN TestBackoff --- PASS: TestBackoff (0.00s) PASS ok k8s.io/kops/pkg/backoff 0.004s === RUN TestSSHCredentialStoreOnConfigBase --- PASS: TestSSHCredentialStoreOnConfigBase (0.00s) === RUN TestSSHCredentialStoreOnOwnCFS --- PASS: TestSSHCredentialStoreOnOwnCFS (0.00s) PASS ok k8s.io/kops/pkg/client/simple/vfsclientset 0.057s === RUN TestToAzureVMName === RUN TestToAzureVMName/test_case_0 === RUN TestToAzureVMName/test_case_1 --- PASS: TestToAzureVMName (0.00s) --- PASS: TestToAzureVMName/test_case_0 (0.00s) --- PASS: TestToAzureVMName/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/cloudinstances 0.025s === RUN TestSetClusterBadInput --- PASS: TestSetClusterBadInput (0.00s) === RUN TestSetClusterFields --- PASS: TestSetClusterFields (0.00s) === RUN TestSetCiliumFields --- PASS: TestSetCiliumFields (0.00s) === RUN TestSetInstanceGroupsBadInput --- PASS: TestSetInstanceGroupsBadInput (0.00s) === RUN TestSetInstanceGroupsFields --- PASS: TestSetInstanceGroupsFields (0.00s) === RUN TestUnsetClusterBadInput --- PASS: TestUnsetClusterBadInput (0.00s) === RUN TestUnsetClusterFields --- PASS: TestUnsetClusterFields (0.00s) === RUN TestUnsetCiliumFields --- PASS: TestUnsetCiliumFields (0.00s) === RUN TestUnsetInstanceGroupsBadInput --- PASS: TestUnsetInstanceGroupsBadInput (0.00s) === RUN TestUnsetInstanceGroupsFields --- PASS: TestUnsetInstanceGroupsFields (0.00s) PASS ok k8s.io/kops/pkg/commands 0.088s === RUN TestGetStructVal --- PASS: TestGetStructVal (0.00s) === RUN TestWrongStructField --- PASS: TestWrongStructField (0.00s) PASS ok k8s.io/kops/pkg/configbuilder 0.028s === RUN Test_Diff_1 --- PASS: Test_Diff_1 (0.00s) === RUN Test_Diff_2 --- PASS: Test_Diff_2 (0.00s) === RUN Test_Diff_3 --- PASS: Test_Diff_3 (0.00s) === RUN Test_Diff_ChangedLine --- PASS: Test_Diff_ChangedLine (0.00s) === RUN Test_Diff_4 --- PASS: Test_Diff_4 (0.00s) PASS ok k8s.io/kops/pkg/diff 0.005s === RUN TestIsGossipHostname --- PASS: TestIsGossipHostname (0.00s) PASS ok k8s.io/kops/pkg/dns 0.006s === RUN TestHasExtraFields --- PASS: TestHasExtraFields (0.01s) PASS ok k8s.io/kops/pkg/edit 0.042s === RUN TestFlagToFalse I0216 02:05:56.230916 48848 featureflag_test.go:32] Created flag Unittest1 I0216 02:05:56.230988 48848 featureflag.go:160] FeatureFlag "UnitTest1"=false I0216 02:05:56.231003 48848 featureflag.go:160] FeatureFlag "UnitTest1"=true --- PASS: TestFlagToFalse (0.00s) === RUN TestSetenv I0216 02:05:56.231094 48848 featureflag.go:160] FeatureFlag "UnitTest2"=false --- PASS: TestSetenv (0.00s) PASS ok k8s.io/kops/pkg/featureflag 0.005s === RUN TestBuildKCMFlags --- PASS: TestBuildKCMFlags (0.00s) === RUN TestKubeletConfigSpec --- PASS: TestKubeletConfigSpec (0.00s) === RUN TestBuildAPIServerFlags --- PASS: TestBuildAPIServerFlags (0.00s) PASS ok k8s.io/kops/pkg/flagbuilder 0.031s === RUN TestRenderInstanceGroupZones W0216 02:05:56.251339 48851 instancegroup.go:42] error fetching zones for instancegroup: cannot find subnet "badsubnet" (declared in instance group "", not found in cluster) --- PASS: TestRenderInstanceGroupZones (0.00s) === RUN TestRenderInstanceGroupSubnets --- PASS: TestRenderInstanceGroupSubnets (0.00s) PASS ok k8s.io/kops/pkg/formatter 0.026s === RUN TestWarmPoolOnlyRoll I0216 02:06:09.773890 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.773994 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.774019 49423 instancegroups.go:112] deleting warm pool instance "node-1" --- PASS: TestWarmPoolOnlyRoll (0.00s) === RUN TestRollingUpdateDisabledSurgeOS rollingupdate_os_test.go:99: Failing in new release PR when build is not yet published --- SKIP: TestRollingUpdateDisabledSurgeOS (0.00s) === RUN TestRollingUpdateAllNeedUpdate I0216 02:06:09.774398 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.774417 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.775555 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.775585 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.780803 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.780868 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.780894 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.780934 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:09.783636 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:09.784608 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.784643 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.785790 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.785820 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.791047 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.791242 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:09.798355 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:09.798408 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.799558 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.799592 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.804807 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.804856 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.804881 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.804897 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:09.806067 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:09.806871 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:09.806907 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.808025 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.808057 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.813265 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.813333 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:09.814317 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:09.814359 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.815490 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.815518 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.820720 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.820784 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:09.821711 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:09.821750 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.822877 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.822906 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.828106 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.828134 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.828156 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.828176 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:09.829222 49423 instancegroups.go:407] Draining the node: "node-2a.local". I0216 02:06:09.830019 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:09.830057 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.831210 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.831235 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.836436 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.836482 49423 instancegroups.go:407] Draining the node: "node-2b.local". I0216 02:06:09.837106 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:09.837133 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.838282 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.838306 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.843498 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.843532 49423 instancegroups.go:407] Draining the node: "node-2c.local". I0216 02:06:09.844094 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:09.844120 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.845267 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.845297 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.850505 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.850533 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdate (0.08s) === RUN TestRollingUpdateAllNeedUpdateCloudonly I0216 02:06:09.850929 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.850952 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.852075 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.852128 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.852168 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.852185 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.852213 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.853364 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.853396 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.853406 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:09.853425 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.854543 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.854574 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.854614 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.854630 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:09.854658 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.855807 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.855839 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.855848 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:09.855866 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.856982 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.857027 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.857045 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:09.857073 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.858197 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.858223 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.858269 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.858287 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:09.858312 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.859433 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.859482 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.859505 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:09.859526 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.860674 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. W0216 02:06:09.860711 49423 instancegroups.go:404] Not draining cluster nodes as 'cloudonly' flag is set. I0216 02:06:09.860724 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:09.860750 49423 instancegroups.go:443] waiting for 1ms after terminating instance W0216 02:06:09.861878 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0216 02:06:09.861909 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateCloudonly (0.01s) === RUN TestRollingUpdateAllNeedUpdateNoFailOnValidate I0216 02:06:09.862276 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.862312 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.863476 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.863508 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.863523 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.863579 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.863599 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.863611 49423 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.863621 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:09.864185 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:09.864792 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.864817 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.865942 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.865977 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.866001 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.866043 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:09.866975 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:09.867011 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.868155 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.868182 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.868202 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.868218 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.868231 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.868240 49423 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.868249 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:09.868897 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:09.869395 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:09.869419 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.870543 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.870578 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.870601 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.870653 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:09.871562 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:09.871599 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.872761 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.872795 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.872820 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.872865 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:09.873766 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:09.873806 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.874946 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.874982 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.875007 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.875028 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.875054 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.875075 49423 instancegroups.go:486] Cluster validation failed, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.875092 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:09.876245 49423 instancegroups.go:407] Draining the node: "node-2a.local". I0216 02:06:09.877211 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:09.877249 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.878409 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.878450 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.878474 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.878518 49423 instancegroups.go:407] Draining the node: "node-2b.local". I0216 02:06:09.879463 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:09.879501 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.880696 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.880786 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.880817 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.880860 49423 instancegroups.go:407] Draining the node: "node-2c.local". I0216 02:06:09.882293 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:09.882335 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.883481 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.883509 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. W0216 02:06:09.883523 49423 instancegroups.go:486] Cluster validation failed after terminating instance, proceeding since fail-on-validate is set to false: cluster did not validate within a duration of "0s" I0216 02:06:09.883537 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateAllNeedUpdateNoFailOnValidate (0.02s) === RUN TestRollingUpdateNoneNeedUpdate I0216 02:06:09.883804 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdate (0.00s) === RUN TestRollingUpdateNoneNeedUpdateWithForce I0216 02:06:09.884031 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.884055 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.885146 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.885180 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.890407 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.890494 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.890523 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.890542 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:09.891460 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:09.892279 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.892319 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.893460 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.893491 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.898694 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.898754 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:09.899554 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:09.899584 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.900828 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.900865 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.906072 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.906121 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.906146 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.906165 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:09.907365 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:09.908088 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:09.908130 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.909568 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.909597 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.914818 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.914888 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:09.915827 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:09.915865 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.917000 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.917028 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.922215 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.922268 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:09.923164 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:09.923197 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.924329 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.924359 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.929594 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.929626 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.929649 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.929662 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:09.930317 49423 instancegroups.go:407] Draining the node: "node-2a.local". I0216 02:06:09.930812 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:09.930839 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.931966 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.931994 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.937196 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.937273 49423 instancegroups.go:407] Draining the node: "node-2b.local". I0216 02:06:09.938265 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:09.938306 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.939932 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.939965 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.945174 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.945312 49423 instancegroups.go:407] Draining the node: "node-2c.local". I0216 02:06:09.946445 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:09.946481 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.947582 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.947604 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.952802 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.952847 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNoneNeedUpdateWithForce (0.07s) === RUN TestRollingUpdateEmptyGroup I0216 02:06:09.953063 49423 rollingupdate.go:97] Cloud Instance Group length is zero. Not doing a rolling-update. --- PASS: TestRollingUpdateEmptyGroup (0.00s) === RUN TestRollingUpdateUnknownRole --- PASS: TestRollingUpdateUnknownRole (0.00s) === RUN TestRollingUpdateAllNeedUpdateFailsValidation I0216 02:06:09.953589 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.953619 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.954765 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.954801 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:09.954820 49423 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateFailsValidation (0.00s) === RUN TestRollingUpdateAllNeedUpdateErrorsValidation I0216 02:06:09.955187 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.955214 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.956333 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.956358 49423 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0216 02:06:09.956377 49423 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateAllNeedUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateFailsValidation I0216 02:06:09.956593 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.956619 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:09.956637 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:09.956666 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0216 02:06:09.956687 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateFailsValidation (0.00s) === RUN TestRollingUpdateNodes1NeedsUpdateErrorsValidation I0216 02:06:09.956843 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.956865 49423 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0216 02:06:09.956883 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:09.956899 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0216 02:06:09.956915 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNodes1NeedsUpdateErrorsValidation (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneMaster I0216 02:06:09.957180 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.957207 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.958339 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.958373 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.963572 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.963660 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.963691 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.963708 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:09.964516 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:09.965296 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.965335 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.966458 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.966489 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:09.966502 49423 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterFailsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneMaster I0216 02:06:09.966766 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:09.966785 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.967893 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.967921 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.973140 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.973209 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.973227 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.973237 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:09.973859 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:09.974384 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:09.974409 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.975528 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.975560 49423 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0216 02:06:09.975577 49423 instancegroups.go:482] Cluster did not validate within 0s --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneMaster (0.01s) === RUN TestRollingUpdateNonRelatedInstanceGroupFailure I0216 02:06:09.975916 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.975939 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.975950 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:09.976928 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:09.977683 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:09.977712 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.978849 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.978882 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.984126 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.984207 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:09.985231 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:09.985271 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.986442 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.986478 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.991689 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.991745 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:09.992684 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:09.992718 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:09.993891 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.993932 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:09.999151 49423 instancegroups.go:510] Cluster validated. I0216 02:06:09.999194 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateNonRelatedInstanceGroupFailure (0.02s) === RUN TestRollingUpdateRelatedInstanceGroupFailure I0216 02:06:09.999617 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.999647 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:09.999668 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:09.999687 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0216 02:06:09.999707 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateRelatedInstanceGroupFailure (0.00s) === RUN TestRollingUpdateMasterGroupFailure I0216 02:06:09.999970 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:09.999997 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:10.000014 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:10.000031 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0216 02:06:10.000050 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMasterGroupFailure (0.00s) === RUN TestRollingUpdateValidationErrorInstanceGroupNil I0216 02:06:10.000351 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.000376 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:10.000392 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:10.000413 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster: cluster did not validate within a duration of "0s" I0216 02:06:10.000431 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidationErrorInstanceGroupNil (0.00s) === RUN TestRollingUpdateClusterFailsValidationAfterOneNode I0216 02:06:10.000600 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.000637 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.000654 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:10.001821 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.002689 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.002721 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.003895 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.003944 49423 instancegroups.go:530] Cluster did not pass validation within deadline: testing failure. E0216 02:06:10.003980 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:10.004026 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0216 02:06:10.004041 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterFailsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateClusterErrorsValidationAfterOneNode I0216 02:06:10.004228 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.004246 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.004255 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:10.005023 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.005528 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.005551 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.006685 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.006724 49423 instancegroups.go:520] Cluster did not validate within deadline: testing validation error. E0216 02:06:10.006745 49423 instancegroups.go:482] Cluster did not validate within 0s E0216 02:06:10.006772 49423 rollingupdate.go:201] failed to roll InstanceGroup "node-1": error validating cluster after terminating instance: cluster did not validate within a duration of "0s" I0216 02:06:10.006791 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateClusterErrorsValidationAfterOneNode (0.00s) === RUN TestRollingUpdateFlappingValidation I0216 02:06:10.007125 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:10.007166 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.008335 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.008380 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.009504 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.014708 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.015829 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.021037 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.022157 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.027388 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.027480 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.027512 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.027536 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:10.028436 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:10.029266 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:10.029305 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.030433 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.030466 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.035668 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.035727 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:10.036747 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:10.036781 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.037908 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.037960 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.043204 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.043256 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.043285 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.043299 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:10.044379 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.045156 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.045188 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.046329 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.046362 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.051556 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.051611 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.052513 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.052545 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.053690 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.053723 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.058965 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.059033 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:10.059977 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:10.060010 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.061166 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.061224 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.066450 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.066488 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.066516 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.066556 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:10.067686 49423 instancegroups.go:407] Draining the node: "node-2a.local". I0216 02:06:10.068464 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:10.068497 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.069654 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.069693 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.074937 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.074996 49423 instancegroups.go:407] Draining the node: "node-2b.local". I0216 02:06:10.075700 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:10.075725 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.076865 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.076896 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.082147 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.082237 49423 instancegroups.go:407] Draining the node: "node-2c.local". I0216 02:06:10.083340 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:10.083380 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.084528 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.084581 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.089795 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.089839 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateFlappingValidation (0.08s) === RUN TestRollingUpdateValidatesAfterBastion I0216 02:06:10.090118 49423 instancegroups.go:600] Stopping instance "bastion-1a", in group "bastion-1" (this may take a while). I0216 02:06:10.090141 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.091258 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.091288 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.092420 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.093561 49423 instancegroups.go:533] Cluster did not pass validation, will retry in "1ms": testing failure. I0216 02:06:10.094683 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.099883 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.099957 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.099976 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.099988 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:10.100570 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:10.101117 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:10.101144 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.102271 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.102299 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.108514 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.108585 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:10.109543 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:10.109582 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.110733 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.110761 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.115904 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.115941 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.115959 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.115971 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:10.116782 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.117594 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.117622 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.118759 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.118795 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.123999 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.124073 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.124842 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.124871 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.125979 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.126005 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.131220 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.131282 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:10.132038 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:10.132066 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.133187 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.133217 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.138468 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.138511 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.138525 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.138536 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:10.139310 49423 instancegroups.go:407] Draining the node: "node-2a.local". I0216 02:06:10.139855 49423 instancegroups.go:598] Stopping instance "node-2a", node "node-2a.local", in group "node-2" (this may take a while). I0216 02:06:10.139882 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.141017 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.141042 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.146218 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.146277 49423 instancegroups.go:407] Draining the node: "node-2b.local". I0216 02:06:10.147313 49423 instancegroups.go:598] Stopping instance "node-2b", node "node-2b.local", in group "node-2" (this may take a while). I0216 02:06:10.147351 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.148502 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.148528 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.153708 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.153758 49423 instancegroups.go:407] Draining the node: "node-2c.local". I0216 02:06:10.154490 49423 instancegroups.go:598] Stopping instance "node-2c", node "node-2c.local", in group "node-2" (this may take a while). I0216 02:06:10.154518 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.155650 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.155679 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.160867 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.160898 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateValidatesAfterBastion (0.07s) === RUN TestAddAnnotatedNodesToNeedsUpdate --- PASS: TestAddAnnotatedNodesToNeedsUpdate (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateCloudonly --- PASS: TestAddAnnotatedNodesToNeedsUpdateCloudonly (0.00s) === RUN TestAddAnnotatedNodesToNeedsUpdateNodesMissing --- PASS: TestAddAnnotatedNodesToNeedsUpdateNodesMissing (0.00s) === RUN TestRollingUpdateTaintAllButOneNeedUpdate I0216 02:06:10.161479 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.161497 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.161506 49423 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0216 02:06:10.162066 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.162967 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.162996 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.164122 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.164150 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.169344 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.169390 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.170096 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.170124 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.171244 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.171270 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.178367 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.178409 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateTaintAllButOneNeedUpdate (0.02s) === RUN TestRollingUpdateMaxSurgeIgnoredForMaster I0216 02:06:10.178661 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.178681 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.178690 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:10.179298 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:10.179833 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:10.179858 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.180973 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.180997 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.186222 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.186297 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:10.187124 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:10.187166 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.188331 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.188355 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.193581 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.193617 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeIgnoredForMaster (0.02s) === RUN TestRollingUpdateDisabled I0216 02:06:10.194097 49423 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled W0216 02:06:10.194140 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0216 02:06:10.194159 49423 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled W0216 02:06:10.194170 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0216 02:06:10.194184 49423 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled W0216 02:06:10.194194 49423 instancegroups.go:475] Not validating cluster as cloudonly flag is set. I0216 02:06:10.194213 49423 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0216 02:06:10.194243 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabled (0.00s) === RUN TestRollingUpdateDisabledSurge I0216 02:06:10.194489 49423 instancegroups.go:577] Detaching instance "bastion-1a", in group "bastion-1". I0216 02:06:10.194550 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:10.195669 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.195703 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.200921 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.200958 49423 instancegroups.go:192] Rolling updates for InstanceGroup bastion-1 are disabled I0216 02:06:10.201061 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.201078 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.201088 49423 instancegroups.go:318] Tainting 2 nodes in "master-1" instancegroup. I0216 02:06:10.201875 49423 instancegroups.go:192] Rolling updates for InstanceGroup master-1 are disabled I0216 02:06:10.201898 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.201912 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.201920 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:10.202596 49423 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0216 02:06:10.202647 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:10.203803 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.203831 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.209745 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.209802 49423 instancegroups.go:192] Rolling updates for InstanceGroup node-1 are disabled I0216 02:06:10.209820 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.209839 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.209854 49423 instancegroups.go:318] Tainting 3 nodes in "node-2" instancegroup. I0216 02:06:10.210819 49423 instancegroups.go:575] Detaching instance "node-2c", node "node-2c.local", in group "node-2". I0216 02:06:10.210870 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:10.212047 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.212078 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:10.217319 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.217368 49423 instancegroups.go:192] Rolling updates for InstanceGroup node-2 are disabled I0216 02:06:10.217390 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDisabledSurge (0.02s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdate I0216 02:06:10.217775 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.217807 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.217822 49423 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0216 02:06:10.221497 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.222157 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.222193 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.223338 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.223370 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.223393 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:10.223484 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.224326 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:10.224636 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.224686 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.225847 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.225915 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.326091 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.326241 49423 instancegroups.go:407] Draining the node: "node-1e.local". I0216 02:06:10.326301 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:10.327468 49423 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0216 02:06:10.327732 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:10.327769 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.328942 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.328975 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.329015 49423 instancegroups.go:407] Draining the node: "node-1f.local". I0216 02:06:10.329588 49423 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0216 02:06:10.329618 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.330769 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.330823 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.431262 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.431327 49423 instancegroups.go:407] Draining the node: "node-1g.local". I0216 02:06:10.432029 49423 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0216 02:06:10.432065 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.433195 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.433233 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.433252 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdate (0.22s) === RUN TestRollingUpdateMaxUnavailableAllButOneNeedUpdate I0216 02:06:10.433497 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.433528 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.433541 49423 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0216 02:06:10.435388 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.435449 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.436462 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.436660 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.436698 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.437863 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.437906 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.538336 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.538404 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:10.538475 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:10.539716 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:10.539955 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:10.539989 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.541156 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.541207 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.541241 49423 instancegroups.go:407] Draining the node: "node-1e.local". I0216 02:06:10.541907 49423 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0216 02:06:10.541938 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.543090 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.543158 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.643612 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.643675 49423 instancegroups.go:407] Draining the node: "node-1f.local". I0216 02:06:10.644397 49423 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0216 02:06:10.644434 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.645595 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.645651 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.645676 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxUnavailableAllNeedUpdateMaster I0216 02:06:10.645977 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.646007 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.646023 49423 instancegroups.go:318] Tainting 7 nodes in "master-1" instancegroup. I0216 02:06:10.648306 49423 instancegroups.go:407] Draining the node: "master-1a.local". I0216 02:06:10.649099 49423 instancegroups.go:598] Stopping instance "master-1a", node "master-1a.local", in group "master-1" (this may take a while). I0216 02:06:10.649144 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.650262 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.650293 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.650320 49423 instancegroups.go:407] Draining the node: "master-1c.local". I0216 02:06:10.650358 49423 instancegroups.go:407] Draining the node: "master-1b.local". I0216 02:06:10.651313 49423 instancegroups.go:598] Stopping instance "master-1c", node "master-1c.local", in group "master-1" (this may take a while). I0216 02:06:10.651495 49423 instancegroups.go:598] Stopping instance "master-1b", node "master-1b.local", in group "master-1" (this may take a while). I0216 02:06:10.651522 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.652750 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.652811 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.753275 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.753348 49423 instancegroups.go:407] Draining the node: "master-1e.local". I0216 02:06:10.753395 49423 instancegroups.go:407] Draining the node: "master-1d.local". I0216 02:06:10.754558 49423 instancegroups.go:598] Stopping instance "master-1e", node "master-1e.local", in group "master-1" (this may take a while). I0216 02:06:10.754832 49423 instancegroups.go:598] Stopping instance "master-1d", node "master-1d.local", in group "master-1" (this may take a while). I0216 02:06:10.754884 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.755998 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.756027 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.756050 49423 instancegroups.go:407] Draining the node: "master-1f.local". I0216 02:06:10.756525 49423 instancegroups.go:598] Stopping instance "master-1f", node "master-1f.local", in group "master-1" (this may take a while). I0216 02:06:10.756554 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.757691 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.757760 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.858140 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.858236 49423 instancegroups.go:407] Draining the node: "master-1g.local". I0216 02:06:10.859219 49423 instancegroups.go:598] Stopping instance "master-1g", node "master-1g.local", in group "master-1" (this may take a while). I0216 02:06:10.859268 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.860444 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.860492 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.860506 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxUnavailableAllNeedUpdateMaster (0.21s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdate I0216 02:06:10.860740 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.860763 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.860772 49423 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0216 02:06:10.861832 49423 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0216 02:06:10.861876 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:10.863013 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.863061 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.863075 49423 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0216 02:06:10.863135 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:10.864284 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.864312 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.864338 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:10.864396 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:10.865161 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:10.865334 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:10.865361 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.866495 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.866562 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.967062 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.967156 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:10.967231 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:10.967943 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:10.968614 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:10.968675 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.969888 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.969942 49423 instancegroups.go:510] Cluster validated. I0216 02:06:10.969979 49423 instancegroups.go:407] Draining the node: "node-1e.local". I0216 02:06:10.970978 49423 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0216 02:06:10.971037 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:10.972185 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:10.972281 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.072851 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.072918 49423 instancegroups.go:407] Draining the node: "node-1f.local". I0216 02:06:11.073620 49423 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0216 02:06:11.073659 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.074781 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.074818 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.074850 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeAllButOneNeedUpdate I0216 02:06:11.075146 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.075183 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.075196 49423 instancegroups.go:318] Tainting 6 nodes in "node-1" instancegroup. I0216 02:06:11.076810 49423 instancegroups.go:575] Detaching instance "node-1f", node "node-1f.local", in group "node-1". I0216 02:06:11.076850 49423 instancegroups.go:575] Detaching instance "node-1e", node "node-1e.local", in group "node-1". I0216 02:06:11.076891 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:11.078024 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.078051 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.078076 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.078127 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.078931 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.079107 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.079133 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.080243 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.080287 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.180697 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.180768 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:11.180804 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:11.181386 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:11.181732 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:11.181760 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.182905 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.182966 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.183007 49423 instancegroups.go:407] Draining the node: "node-1e.local". I0216 02:06:11.183854 49423 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0216 02:06:11.183899 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.185062 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.185106 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.285705 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.285793 49423 instancegroups.go:407] Draining the node: "node-1f.local". I0216 02:06:11.286719 49423 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0216 02:06:11.286777 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.287942 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.287974 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.287986 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllButOneNeedUpdate (0.21s) === RUN TestRollingUpdateMaxSurgeGreaterThanNeedUpdate I0216 02:06:11.288212 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.288229 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.288237 49423 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0216 02:06:11.288699 49423 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0216 02:06:11.288721 49423 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". I0216 02:06:11.288742 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:11.289861 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.289893 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.295102 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.295188 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.295220 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.296197 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.296227 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.296438 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.296467 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.297333 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.297361 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.302577 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.302621 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeGreaterThanNeedUpdate (0.01s) === RUN TestRollingUpdateDetachFails I0216 02:06:11.302914 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.302941 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.302955 49423 instancegroups.go:318] Tainting 2 nodes in "node-1" instancegroup. I0216 02:06:11.303686 49423 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". E0216 02:06:11.303740 49423 instancegroups.go:167] Failed to detach instance "node-1b": error detaching instance "node-1b", node "node-1b.local": error detaching instance "node-1b": testing error I0216 02:06:11.303753 49423 instancegroups.go:575] Detaching instance "node-1a", node "node-1a.local", in group "node-1". E0216 02:06:11.303787 49423 instancegroups.go:167] Failed to detach instance "node-1a": error detaching instance "node-1a", node "node-1a.local": error detaching instance "node-1a": testing error I0216 02:06:11.303817 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:11.304937 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.304963 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.310166 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.310223 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.310269 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.311212 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.311241 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.311517 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.311553 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.312361 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.312389 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.317603 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.317634 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateDetachFails (0.01s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached I0216 02:06:11.317862 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.317887 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.317896 49423 instancegroups.go:318] Tainting 4 nodes in "node-1" instancegroup. I0216 02:06:11.318672 49423 instancegroups.go:575] Detaching instance "node-1c", node "node-1c.local", in group "node-1". I0216 02:06:11.318711 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:11.319828 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.319865 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.319882 49423 instancegroups.go:575] Detaching instance "node-1b", node "node-1b.local", in group "node-1". I0216 02:06:11.319936 49423 instancegroups.go:179] waiting for 1ms after detaching instance I0216 02:06:11.321077 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.321103 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.321131 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:11.321182 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.321219 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.322197 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.322230 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.322385 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:11.322409 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.322577 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.322607 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.323342 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.423907 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.423984 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:11.424641 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:11.424679 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.425867 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.425914 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.425928 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateOneAlreadyDetached (0.11s) === RUN TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached I0216 02:06:11.426143 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.426163 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.426171 49423 instancegroups.go:318] Tainting 7 nodes in "node-1" instancegroup. I0216 02:06:11.427422 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.427857 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.427885 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.429032 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.429060 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.429084 49423 instancegroups.go:407] Draining the node: "node-1e.local". I0216 02:06:11.429140 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:11.429894 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:11.430063 49423 instancegroups.go:598] Stopping instance "node-1e", node "node-1e.local", in group "node-1" (this may take a while). I0216 02:06:11.430089 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.431230 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.431274 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.531873 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.531942 49423 instancegroups.go:407] Draining the node: "node-1g.local". I0216 02:06:11.531995 49423 instancegroups.go:407] Draining the node: "node-1f.local". I0216 02:06:11.532594 49423 instancegroups.go:598] Stopping instance "node-1g", node "node-1g.local", in group "node-1" (this may take a while). I0216 02:06:11.532949 49423 instancegroups.go:598] Stopping instance "node-1f", node "node-1f.local", in group "node-1" (this may take a while). I0216 02:06:11.532978 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.534096 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.534129 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.534155 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.534624 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.534649 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.535765 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.535807 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.636384 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.636450 49423 instancegroups.go:407] Draining the node: "node-1d.local". I0216 02:06:11.637074 49423 instancegroups.go:598] Stopping instance "node-1d", node "node-1d.local", in group "node-1" (this may take a while). I0216 02:06:11.637110 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.638257 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.638286 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.638298 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateMaxSurgeAllNeedUpdateMaxAlreadyDetached (0.21s) === RUN TestRollingUpdateOnlyWarmPoolNodes I0216 02:06:11.638502 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.638519 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.638527 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0216 02:06:11.638538 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0216 02:06:11.638546 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0216 02:06:11.638558 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingUpdateOnlyWarmPoolNodes (0.00s) === RUN TestRollingWarmPoolBeforeJoinedNodes I0216 02:06:11.638672 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.638688 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.638695 49423 instancegroups.go:318] Tainting 3 nodes in "node-1" instancegroup. I0216 02:06:11.639248 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-a" I0216 02:06:11.639265 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-b" I0216 02:06:11.639274 49423 instancegroups.go:112] deleting warm pool instance "node-1-wp-c" I0216 02:06:11.639302 49423 instancegroups.go:407] Draining the node: "node-1a.local". I0216 02:06:11.639722 49423 instancegroups.go:598] Stopping instance "node-1a", node "node-1a.local", in group "node-1" (this may take a while). I0216 02:06:11.639742 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.640874 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.640894 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.646078 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.646129 49423 instancegroups.go:407] Draining the node: "node-1b.local". I0216 02:06:11.646678 49423 instancegroups.go:598] Stopping instance "node-1b", node "node-1b.local", in group "node-1" (this may take a while). I0216 02:06:11.646704 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.647845 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.647867 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.653106 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.653138 49423 instancegroups.go:407] Draining the node: "node-1c.local". I0216 02:06:11.653610 49423 instancegroups.go:598] Stopping instance "node-1c", node "node-1c.local", in group "node-1" (this may take a while). I0216 02:06:11.653633 49423 instancegroups.go:443] waiting for 1ms after terminating instance I0216 02:06:11.654770 49423 instancegroups.go:477] Validating the cluster. I0216 02:06:11.654795 49423 instancegroups.go:513] Cluster validated; revalidating in 5ms to make sure it does not flap. I0216 02:06:11.659975 49423 instancegroups.go:510] Cluster validated. I0216 02:06:11.659999 49423 rollingupdate.go:214] Rolling update completed for cluster ""! --- PASS: TestRollingWarmPoolBeforeJoinedNodes (0.02s) === RUN TestSettings === RUN TestSettings/DrainAndTerminate === RUN TestSettings/MaxUnavailable === RUN TestSettings/MaxSurge --- PASS: TestSettings (0.00s) --- PASS: TestSettings/DrainAndTerminate (0.00s) --- PASS: TestSettings/MaxUnavailable (0.00s) --- PASS: TestSettings/MaxSurge (0.00s) === RUN TestMaxUnavailable === RUN TestMaxUnavailable/0_1 === RUN TestMaxUnavailable/0%_1 === RUN TestMaxUnavailable/39%_10 === RUN TestMaxUnavailable/100%_10 --- PASS: TestMaxUnavailable (0.00s) --- PASS: TestMaxUnavailable/0_1 (0.00s) --- PASS: TestMaxUnavailable/0%_1 (0.00s) --- PASS: TestMaxUnavailable/39%_10 (0.00s) --- PASS: TestMaxUnavailable/100%_10 (0.00s) === RUN TestMaxSurge === RUN TestMaxSurge/0_1 === RUN TestMaxSurge/0%_1 === RUN TestMaxSurge/31%_10 === RUN TestMaxSurge/100%_10 --- PASS: TestMaxSurge (0.00s) --- PASS: TestMaxSurge/0_1 (0.00s) --- PASS: TestMaxSurge/0%_1 (0.00s) --- PASS: TestMaxSurge/31%_10 (0.00s) --- PASS: TestMaxSurge/100%_10 (0.00s) === RUN TestAWSDefault --- PASS: TestAWSDefault (0.00s) PASS ok k8s.io/kops/pkg/instancegroups 1.988s === RUN TestWriteToken === RUN TestWriteToken/WriteJson === RUN TestWriteToken/WriteJsonWithIndent --- PASS: TestWriteToken (0.00s) --- PASS: TestWriteToken/WriteJson (0.00s) --- PASS: TestWriteToken/WriteJsonWithIndent (0.00s) PASS ok k8s.io/kops/pkg/jsonutils 0.006s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.00s) PASS ok k8s.io/kops/pkg/k8scodecs 0.017s === RUN TestParse E0216 02:06:09.688953 49428 versions.go:96] unable to parse Kubernetes version "" E0216 02:06:09.689091 49428 versions.go:96] unable to parse Kubernetes version "abc" --- PASS: TestParse (0.00s) === RUN TestIsGTE === RUN TestIsGTE/KV_greater_than_Version === RUN TestIsGTE/KV_greater_than_Version#01 === RUN TestIsGTE/KV_equal_Version === RUN TestIsGTE/KV_equal_Version#01 === RUN TestIsGTE/Version_greater_than_KV === RUN TestIsGTE/Version_greater_than_KV#01 --- PASS: TestIsGTE (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version (0.00s) --- PASS: TestIsGTE/KV_greater_than_Version#01 (0.00s) --- PASS: TestIsGTE/KV_equal_Version (0.00s) --- PASS: TestIsGTE/KV_equal_Version#01 (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV (0.00s) --- PASS: TestIsGTE/Version_greater_than_KV#01 (0.00s) PASS ok k8s.io/kops/pkg/k8sversion 0.010s === RUN TestToVersionedYaml --- PASS: TestToVersionedYaml (0.01s) === RUN TestToVersionedJSON --- PASS: TestToVersionedJSON (0.00s) === RUN TestRewriteAPIGroup --- PASS: TestRewriteAPIGroup (0.00s) PASS ok k8s.io/kops/pkg/kopscodecs 0.038s === RUN TestBuildKubecfg === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster === RUN TestBuildKubecfg/Public_DNS_with_kops_auth_plugin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin === RUN TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port --- PASS: TestBuildKubecfg (0.19s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin (0.02s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_secondary_NLB_port (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_admin_and_CLB_ACM_Certificate (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin_and_with_ACM_certificate (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_without_admin (0.01s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Public_DNS_with_Empty_Master_Name (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster (0.00s) --- PASS: TestBuildKubecfg/Public_DNS_with_kops_auth_plugin (0.00s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_internal_DNS_name_with_admin (0.04s) --- PASS: TestBuildKubecfg/Test_Kube_Config_Data_For_Gossip_cluster_with_admin_and_secondary_NLB_port (0.02s) PASS ok k8s.io/kops/pkg/kubeconfig 0.266s === RUN Test_ProxyFunc --- PASS: Test_ProxyFunc (0.00s) === RUN TestBootstrapUserData --- PASS: TestBootstrapUserData (0.02s) === RUN TestValidateAWSVolumeAllow50ratio --- PASS: TestValidateAWSVolumeAllow50ratio (0.00s) PASS ok k8s.io/kops/pkg/model 0.114s === RUN TestRootVolumeOptimizationFlag --- PASS: TestRootVolumeOptimizationFlag (0.00s) === RUN TestAPIServerAdditionalSecurityGroupsWithNLB === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Master === RUN TestAPIServerAdditionalSecurityGroupsWithNLB/Node --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Bastion (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Master (0.00s) --- PASS: TestAPIServerAdditionalSecurityGroupsWithNLB/Node (0.00s) === RUN TestJoinSuffixes --- PASS: TestJoinSuffixes (0.00s) === RUN TestIAMServiceEC2 --- PASS: TestIAMServiceEC2 (0.00s) === RUN Test_formatAWSIAMStatement === RUN Test_formatAWSIAMStatement/namespace_and_name_without_wildcard === RUN Test_formatAWSIAMStatement/name_contains_wildcard === RUN Test_formatAWSIAMStatement/namespace_contains_wildcard --- PASS: Test_formatAWSIAMStatement (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_and_name_without_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/name_contains_wildcard (0.00s) --- PASS: Test_formatAWSIAMStatement/namespace_contains_wildcard (0.00s) PASS ok k8s.io/kops/pkg/model/awsmodel 0.095s === RUN TestAPILoadBalancerModelBuilder_Build --- PASS: TestAPILoadBalancerModelBuilder_Build (0.00s) === RUN TestSubnetForLoadbalancer --- PASS: TestSubnetForLoadbalancer (0.00s) === RUN TestCloudTagsForInstanceGroup --- PASS: TestCloudTagsForInstanceGroup (0.00s) === RUN TestNetworkModelBuilder_Build --- PASS: TestNetworkModelBuilder_Build (0.00s) === RUN TestResourceGroupModelBuilder_Build --- PASS: TestResourceGroupModelBuilder_Build (0.00s) === RUN TestVMScaleSetModelBuilder_Build --- PASS: TestVMScaleSetModelBuilder_Build (0.00s) === RUN TestGetCapacity === RUN TestGetCapacity/test_case_0 === RUN TestGetCapacity/test_case_1 === RUN TestGetCapacity/test_case_2 === RUN TestGetCapacity/test_case_3 --- PASS: TestGetCapacity (0.00s) --- PASS: TestGetCapacity/test_case_0 (0.00s) --- PASS: TestGetCapacity/test_case_1 (0.00s) --- PASS: TestGetCapacity/test_case_2 (0.00s) --- PASS: TestGetCapacity/test_case_3 (0.00s) === RUN TestGetStorageProfile === RUN TestGetStorageProfile/test_case_0 === RUN TestGetStorageProfile/test_case_1 --- PASS: TestGetStorageProfile (0.00s) --- PASS: TestGetStorageProfile/test_case_0 (0.00s) --- PASS: TestGetStorageProfile/test_case_1 (0.00s) === RUN TestParseImage === RUN TestParseImage/test_case_0 === RUN TestParseImage/test_case_1 === RUN TestParseImage/test_case_2 === RUN TestParseImage/test_case_3 --- PASS: TestParseImage (0.00s) --- PASS: TestParseImage/test_case_0 (0.00s) --- PASS: TestParseImage/test_case_1 (0.00s) --- PASS: TestParseImage/test_case_2 (0.00s) --- PASS: TestParseImage/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/model/azuremodel 0.095s === RUN TestCloudConfigurationOptionsBuilder === RUN TestCloudConfigurationOptionsBuilder/neither === RUN TestCloudConfigurationOptionsBuilder/all_false === RUN TestCloudConfigurationOptionsBuilder/all_true === RUN TestCloudConfigurationOptionsBuilder/os_false === RUN TestCloudConfigurationOptionsBuilder/os_true === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_false,_os_true === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_false === RUN TestCloudConfigurationOptionsBuilder/all_true,_os_true --- PASS: TestCloudConfigurationOptionsBuilder (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/neither (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_false,_os_true (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_false (0.00s) --- PASS: TestCloudConfigurationOptionsBuilder/all_true,_os_true (0.00s) === RUN Test_Build_Containerd_Supported_Version --- PASS: Test_Build_Containerd_Supported_Version (0.00s) === RUN Test_Build_Containerd_Unneeded_Runtime --- PASS: Test_Build_Containerd_Unneeded_Runtime (0.00s) === RUN Test_Build_Containerd_Needed_Runtime --- PASS: Test_Build_Containerd_Needed_Runtime (0.00s) === RUN TestImage I0216 02:06:09.747745 49433 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestImage (0.00s) === RUN Test_Build_KCM_Builder --- PASS: Test_Build_KCM_Builder (0.00s) === RUN Test_Build_KCM_Builder_Change_Duration --- PASS: Test_Build_KCM_Builder_Change_Duration (0.00s) === RUN Test_Build_KCM_Builder_CIDR_Mask_Size === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: === RUN Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/100.64.1.0/11: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/65: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/32:2001:DB8::/65 (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/95: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/96: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/97: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/98: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/99: (0.00s) --- PASS: Test_Build_KCM_Builder_CIDR_Mask_Size/2001:DB8::/100: (0.00s) === RUN TestFeatureGatesKubernetesVersion --- PASS: TestFeatureGatesKubernetesVersion (0.00s) === RUN TestFeatureGatesOverride --- PASS: TestFeatureGatesOverride (0.00s) === RUN Test_Build_Scheduler_Without_PolicyConfigMap --- PASS: Test_Build_Scheduler_Without_PolicyConfigMap (0.00s) === RUN Test_Build_Scheduler_PolicyConfigMap_Supported_Version --- PASS: Test_Build_Scheduler_PolicyConfigMap_Supported_Version (0.00s) PASS ok k8s.io/kops/pkg/model/components 0.074s === RUN Test_RunEtcdManagerBuilder I0216 02:06:09.760084 49434 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunEtcdManagerBuilder/basedir=tests/minimal === RUN Test_RunEtcdManagerBuilder/basedir=tests/interval === RUN Test_RunEtcdManagerBuilder/basedir=tests/proxy === RUN Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings W0216 02:06:09.814472 49434 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 W0216 02:06:09.817375 49434 model.go:348] overriding log level in manifest (embedded etcd manifest), new level is 3 --- PASS: Test_RunEtcdManagerBuilder (0.06s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/interval (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/proxy (0.01s) --- PASS: Test_RunEtcdManagerBuilder/basedir=tests/overwrite_settings (0.01s) PASS ok k8s.io/kops/pkg/model/components/etcdmanager 0.147s === RUN Test_RunKubeApiserverBuilder I0216 02:06:09.760717 49435 featureflag.go:160] FeatureFlag "ImageDigest"=false === RUN Test_RunKubeApiserverBuilder/basedir=tests/minimal --- PASS: Test_RunKubeApiserverBuilder (0.02s) --- PASS: Test_RunKubeApiserverBuilder/basedir=tests/minimal (0.02s) PASS ok k8s.io/kops/pkg/model/components/kubeapiserver 0.103s === RUN Test_RunKubeSchedulerBuilder === RUN Test_RunKubeSchedulerBuilder/basedir=tests/minimal === RUN Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig === RUN Test_RunKubeSchedulerBuilder/basedir=tests/mixing --- PASS: Test_RunKubeSchedulerBuilder (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/minimal (0.02s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/kubeschedulerconfig (0.00s) --- PASS: Test_RunKubeSchedulerBuilder/basedir=tests/mixing (0.00s) PASS ok k8s.io/kops/pkg/model/components/kubescheduler 0.102s === RUN TestDefaultInstanceGroupVolumeSize --- PASS: TestDefaultInstanceGroupVolumeSize (0.00s) PASS ok k8s.io/kops/pkg/model/defaults 0.032s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) === RUN TestPolicyGeneration --- PASS: TestPolicyGeneration (0.00s) === RUN TestEmptyPolicy --- PASS: TestEmptyPolicy (0.00s) PASS ok k8s.io/kops/pkg/model/iam 0.086s === RUN TestServerGroupBuilder I0216 02:06:17.829597 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.843573 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.858436 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.882063 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.952967 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.978436 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.993714 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.012278 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.020496 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.067840 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.075901 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.084840 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.095843 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.103977 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:18.112690 50259 featureflag.go:160] FeatureFlag "ImageDigest"=false --- PASS: TestServerGroupBuilder (0.29s) PASS ok k8s.io/kops/pkg/model/openstackmodel 0.385s === RUN Test_NodeUpTabs --- PASS: Test_NodeUpTabs (0.00s) PASS ok k8s.io/kops/pkg/model/resources 0.030s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetVMSSNameFromProviderID === RUN TestGetVMSSNameFromProviderID/test_case_0 === RUN TestGetVMSSNameFromProviderID/test_case_1 === RUN TestGetVMSSNameFromProviderID/test_case_2 --- PASS: TestGetVMSSNameFromProviderID (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_0 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_1 (0.00s) --- PASS: TestGetVMSSNameFromProviderID/test_case_2 (0.00s) === RUN TestIdentifyNode === RUN TestIdentifyNode/test_case_0 === RUN TestIdentifyNode/test_case_1 --- PASS: TestIdentifyNode (0.00s) --- PASS: TestIdentifyNode/test_case_0 (0.00s) --- PASS: TestIdentifyNode/test_case_1 (0.00s) PASS ok k8s.io/kops/pkg/nodeidentity/azure 0.015s === RUN TestBuildNodeLabels === RUN TestBuildNodeLabels/RoleControlPlane === RUN TestBuildNodeLabels/RoleNode --- PASS: TestBuildNodeLabels (0.00s) --- PASS: TestBuildNodeLabels/RoleControlPlane (0.00s) --- PASS: TestBuildNodeLabels/RoleNode (0.00s) PASS ok k8s.io/kops/pkg/nodelabels 0.029s === RUN TestGenerateCertificate === RUN TestGenerateCertificate/selfsigned === RUN TestGenerateCertificate/client === RUN TestGenerateCertificate/server --- PASS: TestGenerateCertificate (0.01s) --- PASS: TestGenerateCertificate/selfsigned (0.01s) --- PASS: TestGenerateCertificate/client (0.00s) --- PASS: TestGenerateCertificate/server (0.00s) === RUN TestCertificateRoundTrip --- PASS: TestCertificateRoundTrip (0.00s) === RUN TestIssueCert === RUN TestIssueCert/ca === RUN TestIssueCert/client === RUN TestIssueCert/clientOneYear === RUN TestIssueCert/#00 === RUN TestIssueCert/server --- PASS: TestIssueCert (1.21s) --- PASS: TestIssueCert/ca (0.25s) --- PASS: TestIssueCert/client (0.75s) --- PASS: TestIssueCert/clientOneYear (0.21s) --- PASS: TestIssueCert/#00 (0.00s) --- PASS: TestIssueCert/server (0.00s) === RUN TestPrivateKeyRoundTripRSA === RUN TestPrivateKeyRoundTripRSA/rsa === RUN TestPrivateKeyRoundTripRSA/ecdsa --- PASS: TestPrivateKeyRoundTripRSA (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/rsa (0.00s) --- PASS: TestPrivateKeyRoundTripRSA/ecdsa (0.00s) === RUN Test_AWSFingerprint_RsaKey1 --- PASS: Test_AWSFingerprint_RsaKey1 (0.00s) === RUN Test_AWSFingerprint_RsaKeyEncrypted --- PASS: Test_AWSFingerprint_RsaKeyEncrypted (0.00s) === RUN Test_AWSFingerprint_Ed25519Key --- PASS: Test_AWSFingerprint_Ed25519Key (0.00s) === RUN Test_AWSFingerprint_TrickyWhitespace --- PASS: Test_AWSFingerprint_TrickyWhitespace (0.00s) === RUN Test_AWSFingerprint_DsaKey --- PASS: Test_AWSFingerprint_DsaKey (0.00s) === RUN Test_OpenSSHFingerprint_RsaKey1 --- PASS: Test_OpenSSHFingerprint_RsaKey1 (0.00s) PASS ok k8s.io/kops/pkg/pki 1.224s === RUN TestLongDesc --- PASS: TestLongDesc (0.00s) PASS ok k8s.io/kops/pkg/pretty 0.005s === RUN TestAddUntaggedRouteTables I0216 02:06:17.826359 50261 routetable.go:53] DescribeRouteTables: { } W0216 02:06:17.826524 50261 tags.go:67] cluster tag not found on route-table:rtb-1234 I0216 02:06:17.826549 50261 aws.go:226] Skipping route table in VPC, but with wrong cluster tag ("other.example.com") --- PASS: TestAddUntaggedRouteTables (0.00s) === RUN TestListIAMInstanceProfiles I0216 02:06:17.826729 50261 iaminstanceprofile.go:193] ListInstanceProfiles: { } W0216 02:06:17.826848 50261 aws.go:2097] could not find instance profile "__no_entity__.me.example.com". Resource may already have been deleted: NoSuchEntity: No such entity --- PASS: TestListIAMInstanceProfiles (0.00s) === RUN TestListIAMRoles I0216 02:06:17.826971 50261 iamrole.go:89] ListRoles: { } --- PASS: TestListIAMRoles (0.00s) === RUN TestListRouteTables I0216 02:06:17.827139 50261 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["me.example.com"] }] } I0216 02:06:17.827246 50261 routetable.go:53] DescribeRouteTables: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/me.example.com"] }] } --- PASS: TestListRouteTables (0.00s) === RUN TestSharedVolume I0216 02:06:17.827381 50261 volumes.go:33] CreateVolume: { TagSpecifications: [{ ResourceType: "volume", Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "shared" }] }] } I0216 02:06:17.827532 50261 volumes.go:33] CreateVolume: { TagSpecifications: [{ Tags: [{ Key: "kubernetes.io/cluster/me.example.com", Value: "owned" }] }] } I0216 02:06:17.827592 50261 volumes.go:128] DescribeVolumes: { } W0216 02:06:17.827646 50261 aws.go:2250] (new) cluster tag not found on volume:vol-2 aws_test.go:334: 2 --- PASS: TestSharedVolume (0.00s) === RUN TestMatchesElbTags --- PASS: TestMatchesElbTags (0.00s) === RUN TestListVPCs I0216 02:06:17.827849 50261 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:17.828128 50261 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:17.828192 50261 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:17.828292 50261 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:17.828378 50261 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:17.828447 50261 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:17.828482 50261 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:17.828554 50261 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:17.828588 50261 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:17.828634 50261 tags.go:42] CreateTags { Resources: ["vpc-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "legacy.example.com" }] } I0216 02:06:17.828678 50261 tags.go:42] CreateTags { Resources: ["vpc-shared"], Tags: [{ Key: "kubernetes.io/cluster/shared.example.com", Value: "shared" }] } I0216 02:06:17.828719 50261 tags.go:42] CreateTags { Resources: ["vpc-shared-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "shared-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/shared-with-legacy.example.com", Value: "shared" }] } I0216 02:06:17.828778 50261 tags.go:42] CreateTags { Resources: ["vpc-owned"], Tags: [{ Key: "kubernetes.io/cluster/owned.example.com", Value: "owned" }] } I0216 02:06:17.828819 50261 tags.go:42] CreateTags { Resources: ["vpc-owned-with-legacy"], Tags: [{ Key: "KubernetesCluster", Value: "owned-with-legacy.example.com" },{ Key: "kubernetes.io/cluster/owned-with-legacy.example.com", Value: "owned" }] } I0216 02:06:17.828877 50261 tags.go:42] CreateTags { Resources: ["vpc-other"], Tags: [{ Key: "KubernetesCluster", Value: "other.example.com" },{ Key: "kubernetes.io/cluster/other.example.com", Value: "shared" }] } I0216 02:06:17.828935 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["mismatch.example.com"] }] } I0216 02:06:17.828993 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/mismatch.example.com"] }] } I0216 02:06:17.829032 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["legacy.example.com"] }] } I0216 02:06:17.829069 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/legacy.example.com"] }] } I0216 02:06:17.829115 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared-with-legacy.example.com"] }] } I0216 02:06:17.829154 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared-with-legacy.example.com"] }] } I0216 02:06:17.829200 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["shared.example.com"] }] } I0216 02:06:17.829236 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/shared.example.com"] }] } I0216 02:06:17.829277 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned-with-legacy.example.com"] }] } I0216 02:06:17.829329 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned-with-legacy.example.com"] }] } I0216 02:06:17.829378 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:KubernetesCluster", Values: ["owned.example.com"] }] } I0216 02:06:17.829416 50261 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag-key", Values: ["kubernetes.io/cluster/owned.example.com"] }] } --- PASS: TestListVPCs (0.00s) PASS ok k8s.io/kops/pkg/resources/aws 0.082s === RUN TestListResourcesAzure --- PASS: TestListResourcesAzure (0.00s) === RUN TestIsOwnedByCluster === RUN TestIsOwnedByCluster/test_case_0 === RUN TestIsOwnedByCluster/test_case_1 === RUN TestIsOwnedByCluster/test_case_2 === RUN TestIsOwnedByCluster/test_case_3 --- PASS: TestIsOwnedByCluster (0.00s) --- PASS: TestIsOwnedByCluster/test_case_0 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_1 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_2 (0.00s) --- PASS: TestIsOwnedByCluster/test_case_3 (0.00s) PASS ok k8s.io/kops/pkg/resources/azure 0.030s === RUN TestNameMatch --- PASS: TestNameMatch (0.00s) === RUN TestMatchesClusterNameWithUUID --- PASS: TestMatchesClusterNameWithUUID (0.00s) PASS ok k8s.io/kops/pkg/resources/gce 0.083s === RUN TestEscapeCommand --- PASS: TestEscapeCommand (0.00s) === RUN TestRawManifest --- PASS: TestRawManifest (0.00s) === RUN TestRawMixedManifest --- PASS: TestRawMixedManifest (0.00s) === RUN TestKeyPairOnlyManifest --- PASS: TestKeyPairOnlyManifest (0.00s) === RUN TestUnitFileExtensionValid === RUN TestUnitFileExtensionValid/valid_extension:_.automount === RUN TestUnitFileExtensionValid/valid_extension:_.device === RUN TestUnitFileExtensionValid/valid_extension:_.mount === RUN TestUnitFileExtensionValid/valid_extension:_.path === RUN TestUnitFileExtensionValid/valid_extension:_.scope === RUN TestUnitFileExtensionValid/valid_extension:_.service === RUN TestUnitFileExtensionValid/valid_extension:_.slice === RUN TestUnitFileExtensionValid/valid_extension:_.socket === RUN TestUnitFileExtensionValid/valid_extension:_.swap === RUN TestUnitFileExtensionValid/valid_extension:_.target === RUN TestUnitFileExtensionValid/valid_extension:_.timer === RUN TestUnitFileExtensionValid/invalid_extension:_(no_extension) === RUN TestUnitFileExtensionValid/invalid_extension:_.not-valid --- PASS: TestUnitFileExtensionValid (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.automount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.device (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.mount (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.path (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.scope (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.service (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.slice (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.socket (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.swap (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.target (0.00s) --- PASS: TestUnitFileExtensionValid/valid_extension:_.timer (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_(no_extension) (0.00s) --- PASS: TestUnitFileExtensionValid/invalid_extension:_.not-valid (0.00s) PASS ok k8s.io/kops/pkg/systemd 0.007s === RUN TestTruncateString === RUN TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false === RUN TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true === RUN TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false --- PASS: TestTruncateString (0.00s) --- PASS: TestTruncateString/input:foo/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_33_characters_long/maxLength:64/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:false (0.00s) --- PASS: TestTruncateString/input:this_string_is_longer_it_is_46_characters_long/maxLength:32/alwaysAddHash:true (0.00s) --- PASS: TestTruncateString/input:this_string_is_even_longer_due_to_extreme_verbosity_it_is_in_fact_84_characters_long/maxLength:64/alwaysAddHash:false (0.00s) PASS ok k8s.io/kops/pkg/truncate 0.006s === RUN TestJoin --- PASS: TestJoin (0.00s) PASS ok k8s.io/kops/pkg/urls 0.004s === RUN TestRoundTrip --- PASS: TestRoundTrip (0.00s) PASS ok k8s.io/kops/pkg/util/stringorslice 0.004s === RUN Test_Overlap --- PASS: Test_Overlap (0.00s) === RUN Test_BelongsTo --- PASS: Test_BelongsTo (0.00s) === RUN Test_SplitInto8 --- PASS: Test_SplitInto8 (0.00s) PASS ok k8s.io/kops/pkg/util/subnet 0.003s === RUN TestRenderGeneralOK --- PASS: TestRenderGeneralOK (0.01s) === RUN TestRenderMissingValue --- PASS: TestRenderMissingValue (0.00s) === RUN TestRenderIndent --- PASS: TestRenderIndent (0.00s) === RUN TestRenderChannelFunctions --- PASS: TestRenderChannelFunctions (0.00s) === RUN TestRenderSnippet --- PASS: TestRenderSnippet (0.00s) === RUN TestRenderContext --- PASS: TestRenderContext (0.00s) === RUN TestAllowForMissingVars --- PASS: TestAllowForMissingVars (0.00s) === RUN TestRenderIntegration --- PASS: TestRenderIntegration (0.00s) PASS ok k8s.io/kops/pkg/util/templater 0.062s === RUN Test_ValidateCloudGroupMissing --- PASS: Test_ValidateCloudGroupMissing (0.00s) === RUN Test_ValidateNodesNotEnough --- PASS: Test_ValidateNodesNotEnough (0.00s) === RUN Test_ValidateDetachedNodesDontCount --- PASS: Test_ValidateDetachedNodesDontCount (0.00s) === RUN Test_ValidateNodeNotReady --- PASS: Test_ValidateNodeNotReady (0.00s) === RUN Test_ValidateMastersNotEnough --- PASS: Test_ValidateMastersNotEnough (0.00s) === RUN Test_ValidateMasterNotReady --- PASS: Test_ValidateMasterNotReady (0.00s) === RUN Test_ValidateMasterStaticPods --- PASS: Test_ValidateMasterStaticPods (0.00s) === RUN Test_ValidateNoPodFailures --- PASS: Test_ValidateNoPodFailures (0.00s) === RUN Test_ValidatePodFailure === RUN Test_ValidatePodFailure/pending-node-kube-system === RUN Test_ValidatePodFailure/pending-node-kube-system#01 === RUN Test_ValidatePodFailure/pending-node-kube-system#02 === RUN Test_ValidatePodFailure/pending-node-otherNamespace === RUN Test_ValidatePodFailure/pending-node-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-node-otherNamespace#02 === RUN Test_ValidatePodFailure/pending-cluster-kube-system === RUN Test_ValidatePodFailure/pending-cluster-kube-system#01 === RUN Test_ValidatePodFailure/pending-cluster-kube-system#02 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/pending-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-node-kube-system === RUN Test_ValidatePodFailure/notready-node-kube-system#01 === RUN Test_ValidatePodFailure/notready-node-kube-system#02 === RUN Test_ValidatePodFailure/notready-node-otherNamespace === RUN Test_ValidatePodFailure/notready-node-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-node-otherNamespace#02 === RUN Test_ValidatePodFailure/notready-cluster-kube-system === RUN Test_ValidatePodFailure/notready-cluster-kube-system#01 === RUN Test_ValidatePodFailure/notready-cluster-kube-system#02 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/notready-cluster-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-node-kube-system === RUN Test_ValidatePodFailure/unknown-node-kube-system#01 === RUN Test_ValidatePodFailure/unknown-node-kube-system#02 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-node-otherNamespace#02 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#01 === RUN Test_ValidatePodFailure/unknown-cluster-kube-system#02 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 === RUN Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 --- PASS: Test_ValidatePodFailure (0.01s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/pending-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/notready-cluster-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-node-otherNamespace#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-kube-system#02 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#01 (0.00s) --- PASS: Test_ValidatePodFailure/unknown-cluster-otherNamespace#02 (0.00s) === RUN Test_ValidateBastionNodes === RUN Test_ValidateBastionNodes/instancegroup's_nodes_not_ready === RUN Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready --- PASS: Test_ValidateBastionNodes (0.00s) --- PASS: Test_ValidateBastionNodes/instancegroup's_nodes_not_ready (0.00s) --- PASS: Test_ValidateBastionNodes/bastion_instancegroup_nodes_not_ready (0.00s) === RUN Test_ValidateDetachedNodesNotValidated --- PASS: Test_ValidateDetachedNodesNotValidated (0.00s) PASS ok k8s.io/kops/pkg/validation 0.091s === RUN TestUnmarshalMetadata --- PASS: TestUnmarshalMetadata (0.00s) === RUN TestGetInternalIP --- PASS: TestGetInternalIP (0.00s) === RUN TestGetSeeds --- PASS: TestGetSeeds (0.00s) PASS ok k8s.io/kops/protokube/pkg/gossip/azure 0.005s === RUN TestRemovesDuplicateGuardedBlocks W0216 02:06:26.270328 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270393 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270402 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270409 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270417 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270424 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270432 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270439 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270446 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270457 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270469 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270477 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270491 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270498 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.270505 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.270512 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274142 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274170 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274236 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274248 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274302 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274313 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274384 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274398 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274468 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274480 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274539 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274550 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274626 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274639 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274724 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274737 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274791 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274807 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274866 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274881 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.274951 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.274964 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275017 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275030 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275082 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275093 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275170 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275196 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275289 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275306 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275397 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275413 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275497 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275513 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275594 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275619 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275709 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275731 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275865 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.275890 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.275986 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276002 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276092 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276109 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276195 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276211 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276299 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276316 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276436 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276458 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276545 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276562 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276653 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276670 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276756 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276773 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276862 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.276879 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.276984 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277002 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277093 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277116 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277214 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277238 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277336 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277353 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277447 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277473 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277564 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277580 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277678 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277696 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277809 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277829 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.277936 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.277956 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278045 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278063 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278156 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278174 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278260 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278279 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278402 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278421 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278547 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278565 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278644 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278662 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278743 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278761 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278859 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278879 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.278967 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.278986 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279091 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279113 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279197 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279215 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279299 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279316 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279405 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279423 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279505 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279523 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279613 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279630 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279718 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279737 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279820 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279837 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.279922 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.279940 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280023 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280041 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280127 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280145 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280225 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280243 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280330 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280348 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280433 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280451 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280537 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280555 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280648 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280667 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280787 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280806 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280891 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.280908 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.280993 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281010 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281115 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281134 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281216 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281233 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281319 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281337 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281418 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281435 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281517 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281534 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281620 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281637 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281725 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281743 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281832 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281850 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.281959 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.281977 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282061 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282079 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282165 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282183 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282266 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282285 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282372 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282390 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282479 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282497 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282582 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282600 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282690 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282708 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282791 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282808 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.282892 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.282910 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283017 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283036 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283121 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283138 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283216 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283233 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283348 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283368 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283454 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283473 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283566 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283585 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283675 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283693 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283786 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283805 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.283890 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.283907 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284020 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284040 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284126 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284143 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284246 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284265 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284353 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284371 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284463 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284481 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.284570 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.284589 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRemovesDuplicateGuardedBlocks (0.02s) === RUN TestRecoversFromBadNesting W0216 02:06:26.285521 51721 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0216 02:06:26.285537 51721 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0216 02:06:26.285549 51721 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0216 02:06:26.285561 51721 hosts.go:144] /etc/hosts guard-block end seen before guard-block start; will ignore end W0216 02:06:26.285572 51721 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0216 02:06:26.285583 51721 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0216 02:06:26.285593 51721 hosts.go:131] /etc/hosts guard-block begin seen while in guard block; will ignore W0216 02:06:26.285635 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285652 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285666 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.285678 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285690 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.285702 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285716 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.285728 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285740 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.285750 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285762 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285774 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285786 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285798 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.285810 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.285822 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303388 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303419 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303485 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303509 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303567 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303579 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303635 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303647 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303700 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303711 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303760 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303776 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303870 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303883 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.303936 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.303947 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304014 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304027 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304085 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304096 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304156 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304167 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304229 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304240 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304307 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304319 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304386 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304419 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304476 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304502 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304558 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304575 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304629 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304641 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304693 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304704 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304758 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304769 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304833 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304847 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304908 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304920 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.304968 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.304983 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305041 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305057 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305146 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305166 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305241 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305258 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305362 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305379 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305460 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305475 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305568 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305585 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305662 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305679 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305759 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305775 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305881 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305896 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.305983 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.305997 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306078 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306093 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306180 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306193 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306265 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306279 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306361 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306374 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306455 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306469 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306554 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306574 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306669 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306690 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306782 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306799 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.306903 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.306918 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307004 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307017 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307072 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307083 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307146 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307158 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307210 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307222 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307270 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307281 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307368 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307386 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307462 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307475 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307531 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307543 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307597 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307608 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307656 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307667 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307726 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307741 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307791 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307802 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307856 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307870 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.307919 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.307929 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308013 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308025 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308086 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308103 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308160 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308171 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308219 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308229 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308288 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308300 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308348 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308358 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308405 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308416 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308477 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308488 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308549 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308560 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308622 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308634 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308683 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308698 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308752 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308762 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308823 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308834 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308883 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308893 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.308945 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.308973 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309030 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309042 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309094 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309104 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309152 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309163 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309218 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309229 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309278 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309289 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309357 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309369 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309424 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309435 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309487 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309499 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309548 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309564 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309634 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309646 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309700 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309712 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309760 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309770 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309828 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309840 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309900 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309912 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.309970 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.309981 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310029 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310044 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310093 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310103 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310157 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310168 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310221 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310237 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310293 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310304 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310351 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310363 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310431 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310443 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310493 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310504 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310558 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310569 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310616 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310643 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310711 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310723 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310774 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310786 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310833 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310844 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" W0216 02:06:26.310891 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# Begin host entries managed by kops - do not edit" W0216 02:06:26.310902 51721 hosts.go:57] ignoring extra guard line in /etc/hosts: "# End host entries managed by kops" --- PASS: TestRecoversFromBadNesting (0.03s) PASS ok k8s.io/kops/protokube/pkg/gossip/dns/hosts 0.045s === RUN TestGetMetadataFromMetadataServiceReturnsNotFoundError --- PASS: TestGetMetadataFromMetadataServiceReturnsNotFoundError (0.00s) === RUN TestGetMetadataFromMetadataService --- PASS: TestGetMetadataFromMetadataService (0.00s) === RUN TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound --- PASS: TestGetMetadataFromConfigDriveReturnsErrorWhenNoDeviceIsFound (0.00s) === RUN TestGetMetadataFromConfigDrive --- PASS: TestGetMetadataFromConfigDrive (0.00s) === RUN TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound --- PASS: TestGetMetadataReturnsLastErrorWhenNoMetadataWasFound (0.00s) === RUN TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromConfigDriveWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenConfigDriveFails --- PASS: TestGetMetadataFromServiceEndpointWhenConfigDriveFails (0.00s) === RUN TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder --- PASS: TestGetMetadataFromServiceEndpointWhenItIsFirstInSearchOrder (0.00s) === RUN TestGetMetadataFromConfigDriveWhenServiceEndpointFails --- PASS: TestGetMetadataFromConfigDriveWhenServiceEndpointFails (0.00s) PASS ok k8s.io/kops/protokube/pkg/protokube 0.069s === RUN TestSerializeEmptyCluster --- PASS: TestSerializeEmptyCluster (0.00s) PASS ok k8s.io/kops/tests/codecs 0.028s === RUN TestKopsUpgrades --- PASS: TestKopsUpgrades (0.01s) === RUN TestKubernetesUpgrades --- PASS: TestKubernetesUpgrades (0.00s) === RUN TestFindImage --- PASS: TestFindImage (0.00s) === RUN TestRecommendedKubernetesVersion --- PASS: TestRecommendedKubernetesVersion (0.00s) === RUN TestOrdering --- PASS: TestOrdering (0.00s) === RUN TestChannelsSelfConsistent --- PASS: TestChannelsSelfConsistent (0.02s) === RUN TestChannelImages === RUN TestChannelImages/stable-channel === RUN TestChannelImages/alpha-channel --- PASS: TestChannelImages (0.00s) --- PASS: TestChannelImages/stable-channel (0.00s) --- PASS: TestChannelImages/alpha-channel (0.00s) PASS ok k8s.io/kops/tests/integration/channel 0.058s === RUN TestConversionMinimal === RUN TestConversionMinimal/legacy-v1alpha2-v1alpha2 === RUN TestConversionMinimal/v1alpha2-v1alpha3 === RUN TestConversionMinimal/v1alpha3-v1alpha2 --- PASS: TestConversionMinimal (0.02s) --- PASS: TestConversionMinimal/legacy-v1alpha2-v1alpha2 (0.01s) --- PASS: TestConversionMinimal/v1alpha2-v1alpha3 (0.01s) --- PASS: TestConversionMinimal/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionAWS === RUN TestConversionAWS/v1alpha2-v1alpha3 === RUN TestConversionAWS/v1alpha3-v1alpha2 --- PASS: TestConversionAWS (0.01s) --- PASS: TestConversionAWS/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAWS/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionAzure === RUN TestConversionAzure/v1alpha2-v1alpha3 === RUN TestConversionAzure/v1alpha3-v1alpha2 --- PASS: TestConversionAzure (0.01s) --- PASS: TestConversionAzure/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionAzure/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCanal === RUN TestConversionCanal/v1alpha2-v1alpha3 === RUN TestConversionCanal/v1alpha3-v1alpha2 --- PASS: TestConversionCanal (0.01s) --- PASS: TestConversionCanal/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCanal/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionCilium === RUN TestConversionCilium/v1alpha2-v1alpha3 === RUN TestConversionCilium/v1alpha3-v1alpha2 --- PASS: TestConversionCilium (0.01s) --- PASS: TestConversionCilium/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionCilium/v1alpha3-v1alpha2 (0.01s) === RUN TestConversionDO === RUN TestConversionDO/v1alpha2-v1alpha3 === RUN TestConversionDO/v1alpha3-v1alpha2 --- PASS: TestConversionDO (0.01s) --- PASS: TestConversionDO/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionDO/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionGCE === RUN TestConversionGCE/v1alpha2-v1alpha3 === RUN TestConversionGCE/v1alpha3-v1alpha2 --- PASS: TestConversionGCE (0.00s) --- PASS: TestConversionGCE/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionGCE/v1alpha3-v1alpha2 (0.00s) === RUN TestConversionOpenstack === RUN TestConversionOpenstack/v1alpha2-v1alpha3 === RUN TestConversionOpenstack/v1alpha3-v1alpha2 --- PASS: TestConversionOpenstack (0.01s) --- PASS: TestConversionOpenstack/v1alpha2-v1alpha3 (0.00s) --- PASS: TestConversionOpenstack/v1alpha3-v1alpha2 (0.00s) PASS ok k8s.io/kops/tests/integration/conversion 0.101s === RUN Test_tryResourceAsString --- PASS: Test_tryResourceAsString (0.00s) === RUN Test_DryrunTarget_PrintReport --- PASS: Test_DryrunTarget_PrintReport (0.00s) === RUN TestWriteFile I0216 02:06:28.652864 52225 files.go:57] Writing file "/tmp/TestWriteFile1917503771/001/SubDir/test1.tmp" I0216 02:06:28.653022 52225 files.go:113] Changing file mode for "/tmp/TestWriteFile1917503771/001/SubDir/test1.tmp" to -rw-r--r-- --- PASS: TestWriteFile (0.00s) === RUN TestBigInt_Format --- PASS: TestBigInt_Format (0.00s) === RUN TestVFSCAStoreRoundTrip --- PASS: TestVFSCAStoreRoundTrip (0.00s) === RUN TestVFSCAStoreRoundTripWithVault vfs_castore_test.go:161: No vault dev token set --- SKIP: TestVFSCAStoreRoundTripWithVault (0.00s) === RUN TestAddItem === RUN TestAddItem/first_primary === RUN TestAddItem/first_secondary === RUN TestAddItem/firstBigSerial_primary === RUN TestAddItem/firstBigSerial_secondary === RUN TestAddItem/after_primary === RUN TestAddItem/after_secondary === RUN TestAddItem/bigSerialAfter_primary === RUN TestAddItem/bigSerialAfter_secondary === RUN TestAddItem/before_primary === RUN TestAddItem/before_secondary === RUN TestAddItem/first_certonly_primary === RUN TestAddItem/first_certonly_secondary === RUN TestAddItem/after_certonly_primary === RUN TestAddItem/after_certonly_secondary === RUN TestAddItem/before_certonly_primary === RUN TestAddItem/before_certonly_secondary --- PASS: TestAddItem (0.00s) --- PASS: TestAddItem/first_primary (0.00s) --- PASS: TestAddItem/first_secondary (0.00s) --- PASS: TestAddItem/firstBigSerial_primary (0.00s) --- PASS: TestAddItem/firstBigSerial_secondary (0.00s) --- PASS: TestAddItem/after_primary (0.00s) --- PASS: TestAddItem/after_secondary (0.00s) --- PASS: TestAddItem/bigSerialAfter_primary (0.00s) --- PASS: TestAddItem/bigSerialAfter_secondary (0.00s) --- PASS: TestAddItem/before_primary (0.00s) --- PASS: TestAddItem/before_secondary (0.00s) --- PASS: TestAddItem/first_certonly_primary (0.00s) --- PASS: TestAddItem/first_certonly_secondary (0.00s) --- PASS: TestAddItem/after_certonly_primary (0.00s) --- PASS: TestAddItem/after_certonly_secondary (0.00s) --- PASS: TestAddItem/before_certonly_primary (0.00s) --- PASS: TestAddItem/before_certonly_secondary (0.00s) PASS ok k8s.io/kops/upup/pkg/fi 0.035s === RUN TestBootstrapChannelBuilder_BuildTasks I0216 02:06:28.858922 52318 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:28.859293 52318 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:28.859367 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:28.859430 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:28.859517 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:28.859586 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:28.859620 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:28.859668 52318 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:28.859718 52318 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:28.875976 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:28.933815 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.059736 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.154996 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.245799 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.351428 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.428365 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.501527 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:29.608784 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:30.081177 52318 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_BuildTasks (1.27s) === RUN TestBootstrapChannelBuilder_ServiceAccountIAM I0216 02:06:30.126956 52318 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:30.127119 52318 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:30.127182 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:30.127217 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:30.127251 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:30.127278 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:30.127312 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:30.127336 52318 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:30.127356 52318 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:30.127375 52318 featureflag.go:163] Unknown FeatureFlag "+UseServiceAccountExternalPermissions" I0216 02:06:30.128525 52318 zones.go:93] ListHostedZonesPages { } I0216 02:06:30.200072 52318 featureflag.go:163] Unknown FeatureFlag "-UseServiceAccountExternalPermissions" --- PASS: TestBootstrapChannelBuilder_ServiceAccountIAM (0.07s) === RUN TestBootstrapChannelBuilder_AWSCloudController I0216 02:06:30.200287 52318 featureflag.go:160] FeatureFlag "ImageDigest"=false I0216 02:06:30.200439 52318 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:30.200490 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-12345678" } I0216 02:06:30.200526 52318 subnets.go:185] AuthorizeSecurityGroupIngress: { RouteTableId: "rtb-12345678", SubnetId: "subnet-abcdef" } I0216 02:06:30.200561 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-1" } I0216 02:06:30.200586 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-2" } I0216 02:06:30.200605 52318 targetgroups.go:94] CreateTargetGroup { Name: "my-external-tg-3" } I0216 02:06:30.200645 52318 iaminstanceprofile.go:56] CreateInstanceProfile: { InstanceProfileName: "kops-custom-node-role" } I0216 02:06:30.200665 52318 iaminstanceprofile.go:128] AddRoleToInstanceProfile: { InstanceProfileName: "kops-custom-node-role", RoleName: "kops-custom-node-role" } I0216 02:06:30.201849 52318 zones.go:93] ListHostedZonesPages { } --- PASS: TestBootstrapChannelBuilder_AWSCloudController (0.10s) === RUN TestContainerdVersionUrlHash === RUN TestContainerdVersionUrlHash/1.3.4-amd64 === RUN TestContainerdVersionUrlHash/1.3.4-arm64 === RUN TestContainerdVersionUrlHash/1.3.10-amd64 === RUN TestContainerdVersionUrlHash/1.3.10-arm64 === RUN TestContainerdVersionUrlHash/1.4.1-amd64 === RUN TestContainerdVersionUrlHash/1.4.1-arm64 === RUN TestContainerdVersionUrlHash/1.4.4-amd64 === RUN TestContainerdVersionUrlHash/1.4.4-arm64 --- PASS: TestContainerdVersionUrlHash (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.3.10-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrlHash/1.4.4-arm64 (0.00s) === RUN TestContainerdVersionUrl === RUN TestContainerdVersionUrl/1.4.1- === RUN TestContainerdVersionUrl/1.4.1-arm === RUN TestContainerdVersionUrl/-amd64 === RUN TestContainerdVersionUrl/-arm64 === RUN TestContainerdVersionUrl/1.1.1-amd64 === RUN TestContainerdVersionUrl/1.1.1-arm64 === RUN TestContainerdVersionUrl/1.3.5-amd64 === RUN TestContainerdVersionUrl/1.3.5-arm64 === RUN TestContainerdVersionUrl/1.3.4-amd64 === RUN TestContainerdVersionUrl/1.3.4-arm64 === RUN TestContainerdVersionUrl/1.4.1-amd64 === RUN TestContainerdVersionUrl/1.4.1-arm64 === RUN TestContainerdVersionUrl/1.4.3-amd64 === RUN TestContainerdVersionUrl/1.4.3-arm64 --- PASS: TestContainerdVersionUrl (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1- (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionUrl/-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.3.4-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionUrl/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionHash === RUN TestContainerdVersionHash/1.4.1- === RUN TestContainerdVersionHash/1.4.1-arm === RUN TestContainerdVersionHash/-amd64 === RUN TestContainerdVersionHash/-arm64 === RUN TestContainerdVersionHash/1.1.1-amd64 === RUN TestContainerdVersionHash/1.1.1-arm64 === RUN TestContainerdVersionHash/1.3.5-amd64 === RUN TestContainerdVersionHash/1.3.5-arm64 === RUN TestContainerdVersionHash/1.4.1-amd64 === RUN TestContainerdVersionHash/1.4.1-arm64 === RUN TestContainerdVersionHash/1.4.3-amd64 === RUN TestContainerdVersionHash/1.4.3-arm64 --- PASS: TestContainerdVersionHash (0.00s) --- PASS: TestContainerdVersionHash/1.4.1- (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm (0.00s) --- PASS: TestContainerdVersionHash/-amd64 (0.00s) --- PASS: TestContainerdVersionHash/-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.3.5-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.1-arm64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-amd64 (0.00s) --- PASS: TestContainerdVersionHash/1.4.3-arm64 (0.00s) === RUN TestContainerdVersionsHashesAmd64 containerd_test.go:316: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesAmd64 (0.00s) === RUN TestContainerdVersionsHashesArm64 containerd_test.go:331: VERIFY_HASHES not set, won't download & verify containerd hashes --- SKIP: TestContainerdVersionsHashesArm64 (0.00s) === RUN TestDeepValidate_OK --- PASS: TestDeepValidate_OK (0.00s) === RUN TestDeepValidate_NoNodeZones --- PASS: TestDeepValidate_NoNodeZones (0.00s) === RUN TestDeepValidate_NoMasterZones --- PASS: TestDeepValidate_NoMasterZones (0.00s) === RUN TestDeepValidate_BadZone deepvalidate_test.go:57: Zone validation not checked by DeepValidate --- SKIP: TestDeepValidate_BadZone (0.00s) === RUN TestDeepValidate_MixedRegion deepvalidate_test.go:69: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_MixedRegion (0.00s) === RUN TestDeepValidate_RegionAsZone deepvalidate_test.go:83: Region validation not checked by DeepValidate --- SKIP: TestDeepValidate_RegionAsZone (0.00s) === RUN TestDeepValidate_NotIncludedZone --- PASS: TestDeepValidate_NotIncludedZone (0.00s) === RUN TestDeepValidate_DuplicateZones --- PASS: TestDeepValidate_DuplicateZones (0.00s) === RUN TestDeepValidate_ExtraMasterZone --- PASS: TestDeepValidate_ExtraMasterZone (0.00s) === RUN TestDeepValidate_EvenEtcdClusterSize --- PASS: TestDeepValidate_EvenEtcdClusterSize (0.00s) === RUN TestDeepValidate_MissingEtcdMember --- PASS: TestDeepValidate_MissingEtcdMember (0.00s) === RUN TestPopulateClusterSpec_Proxy --- PASS: TestPopulateClusterSpec_Proxy (0.00s) === RUN TestPrecreateDNSNames --- PASS: TestPrecreateDNSNames (0.00s) === RUN TestDockerVersionUrlHash === RUN TestDockerVersionUrlHash/19.03.13-amd64 === RUN TestDockerVersionUrlHash/19.03.13-arm64 === RUN TestDockerVersionUrlHash/18.06.3-amd64 === RUN TestDockerVersionUrlHash/18.06.3-arm64 === RUN TestDockerVersionUrlHash/17.03.1-amd64 === RUN TestDockerVersionUrlHash/17.03.1-arm64 --- PASS: TestDockerVersionUrlHash (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/19.03.13-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-amd64 (0.00s) --- PASS: TestDockerVersionUrlHash/17.03.1-arm64 (0.00s) === RUN TestDockerVersionUrl === RUN TestDockerVersionUrl/19.03.13- === RUN TestDockerVersionUrl/19.03.13-arm === RUN TestDockerVersionUrl/-amd64 === RUN TestDockerVersionUrl/-arm64 === RUN TestDockerVersionUrl/18.06.3-amd64 === RUN TestDockerVersionUrl/18.06.3-arm64 === RUN TestDockerVersionUrl/19.03.13-amd64 === RUN TestDockerVersionUrl/19.03.13-arm64 --- PASS: TestDockerVersionUrl (0.00s) --- PASS: TestDockerVersionUrl/19.03.13- (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm (0.00s) --- PASS: TestDockerVersionUrl/-amd64 (0.00s) --- PASS: TestDockerVersionUrl/-arm64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-amd64 (0.00s) --- PASS: TestDockerVersionUrl/18.06.3-arm64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionUrl/19.03.13-arm64 (0.00s) === RUN TestDockerVersionHash === RUN TestDockerVersionHash/19.03.13- === RUN TestDockerVersionHash/19.03.13-arm === RUN TestDockerVersionHash/-amd64 === RUN TestDockerVersionHash/-arm64 === RUN TestDockerVersionHash/1.1.1-amd64 === RUN TestDockerVersionHash/1.1.1-arm64 === RUN TestDockerVersionHash/19.03.13-amd64 === RUN TestDockerVersionHash/19.03.13-arm64 --- PASS: TestDockerVersionHash (0.00s) --- PASS: TestDockerVersionHash/19.03.13- (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm (0.00s) --- PASS: TestDockerVersionHash/-amd64 (0.00s) --- PASS: TestDockerVersionHash/-arm64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-amd64 (0.00s) --- PASS: TestDockerVersionHash/1.1.1-arm64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-amd64 (0.00s) --- PASS: TestDockerVersionHash/19.03.13-arm64 (0.00s) === RUN TestDockerVersionsHashesAmd64 docker_test.go:245: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesAmd64 (0.00s) === RUN TestDockerVersionsHashesArm64 docker_test.go:260: VERIFY_HASHES not set, won't download & verify docker hashes --- SKIP: TestDockerVersionsHashesArm64 (0.00s) === RUN Test_FindCNIAssetFromEnvironmentVariable --- PASS: Test_FindCNIAssetFromEnvironmentVariable (0.00s) === RUN Test_FindCNIAssetFromDefaults118 --- PASS: Test_FindCNIAssetFromDefaults118 (0.09s) === RUN Test_FindCNIAssetFromDefaults122 --- PASS: Test_FindCNIAssetFromDefaults122 (0.01s) === RUN TestRemoveSharedPrefix --- PASS: TestRemoveSharedPrefix (0.00s) === RUN TestCreateEtcdCluster --- PASS: TestCreateEtcdCluster (0.00s) === RUN TestSetupNetworking W0216 02:06:30.420287 52318 new_cluster.go:1089] flannel UDP mode is not recommended; consider flannel-vxlan instead --- PASS: TestSetupNetworking (0.01s) === RUN TestPopulateCluster_Default_NoError --- PASS: TestPopulateCluster_Default_NoError (0.00s) === RUN TestPopulateCluster_Subnets === RUN TestPopulateCluster_Subnets/100.64.0.0/10 === RUN TestPopulateCluster_Subnets/10.0.0.0/9 === RUN TestPopulateCluster_Subnets/10.0.0.0/8 === RUN TestPopulateCluster_Subnets/::/0 --- PASS: TestPopulateCluster_Subnets (0.01s) --- PASS: TestPopulateCluster_Subnets/100.64.0.0/10 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/9 (0.00s) --- PASS: TestPopulateCluster_Subnets/10.0.0.0/8 (0.00s) --- PASS: TestPopulateCluster_Subnets/::/0 (0.00s) === RUN TestPopulateCluster_Docker_Spec --- PASS: TestPopulateCluster_Docker_Spec (0.00s) === RUN TestPopulateCluster_StorageDefault --- PASS: TestPopulateCluster_StorageDefault (0.00s) === RUN TestPopulateCluster_EvictionHard --- PASS: TestPopulateCluster_EvictionHard (0.00s) === RUN TestPopulateCluster_Custom_CIDR --- PASS: TestPopulateCluster_Custom_CIDR (0.00s) === RUN TestPopulateCluster_IsolateMasters --- PASS: TestPopulateCluster_IsolateMasters (0.00s) === RUN TestPopulateCluster_IsolateMastersFalse --- PASS: TestPopulateCluster_IsolateMastersFalse (0.00s) === RUN TestPopulateCluster_Name_Required --- PASS: TestPopulateCluster_Name_Required (0.00s) === RUN TestPopulateCluster_Zone_Required --- PASS: TestPopulateCluster_Zone_Required (0.00s) === RUN TestPopulateCluster_NetworkCIDR_Required --- PASS: TestPopulateCluster_NetworkCIDR_Required (0.00s) === RUN TestPopulateCluster_NonMasqueradeCIDR_Required --- PASS: TestPopulateCluster_NonMasqueradeCIDR_Required (0.00s) === RUN TestPopulateCluster_CloudProvider_Required --- PASS: TestPopulateCluster_CloudProvider_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidNil_Required --- PASS: TestPopulateCluster_TopologyInvalidNil_Required (0.00s) === RUN TestPopulateCluster_TopologyInvalidValue_Required --- PASS: TestPopulateCluster_TopologyInvalidValue_Required (0.00s) === RUN TestPopulateCluster_BastionInvalidMatchingValues_Required --- PASS: TestPopulateCluster_BastionInvalidMatchingValues_Required (0.00s) === RUN TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required --- PASS: TestPopulateCluster_BastionIdleTimeoutInvalidNegative_Required (0.00s) === RUN TestPopulateCluster_APIServerCount --- PASS: TestPopulateCluster_APIServerCount (0.00s) === RUN TestPopulateCluster_AnonymousAuth --- PASS: TestPopulateCluster_AnonymousAuth (0.00s) === RUN TestPopulateCluster_DockerVersion --- PASS: TestPopulateCluster_DockerVersion (0.00s) === RUN TestPopulateCluster_KubeController_High_Enough_Version --- PASS: TestPopulateCluster_KubeController_High_Enough_Version (0.00s) === RUN TestPopulateInstanceGroup_Name_Required --- PASS: TestPopulateInstanceGroup_Name_Required (0.00s) === RUN TestPopulateInstanceGroup_Role_Required --- PASS: TestPopulateInstanceGroup_Role_Required (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision --- PASS: TestPopulateInstanceGroup_AddTaintsCollision (0.00s) === RUN TestPopulateInstanceGroup_AddTaintsCollision3 --- PASS: TestPopulateInstanceGroup_AddTaintsCollision3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard --- PASS: TestPopulateInstanceGroup_EvictionHard (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard3 --- PASS: TestPopulateInstanceGroup_EvictionHard3 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard4 --- PASS: TestPopulateInstanceGroup_EvictionHard4 (0.00s) === RUN TestPopulateInstanceGroup_EvictionHard2 --- PASS: TestPopulateInstanceGroup_EvictionHard2 (0.00s) === RUN TestPopulateInstanceGroup_AddTaints --- PASS: TestPopulateInstanceGroup_AddTaints (0.00s) === RUN TestMachineArchitecture === RUN TestMachineArchitecture/t2.micro-amd64 === RUN TestMachineArchitecture/t3.micro-amd64 === RUN TestMachineArchitecture/a1.large-arm64 --- PASS: TestMachineArchitecture (0.00s) --- PASS: TestMachineArchitecture/t2.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/t3.micro-amd64 (0.00s) --- PASS: TestMachineArchitecture/a1.large-arm64 (0.00s) === RUN TestRuncVersionUrlHash === RUN TestRuncVersionUrlHash/1.100.0-amd64 === RUN TestRuncVersionUrlHash/1.100.0-arm64 === RUN TestRuncVersionUrlHash/1.1.0-amd64 === RUN TestRuncVersionUrlHash/1.1.0-arm64 --- PASS: TestRuncVersionUrlHash (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrlHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionUrl === RUN TestRuncVersionUrl/1.1.0- === RUN TestRuncVersionUrl/1.1.0-arm === RUN TestRuncVersionUrl/-amd64 === RUN TestRuncVersionUrl/-arm64 === RUN TestRuncVersionUrl/1.0.0-amd64 === RUN TestRuncVersionUrl/1.0.0-arm64 === RUN TestRuncVersionUrl/1.1.0-amd64 === RUN TestRuncVersionUrl/1.1.0-arm64 --- PASS: TestRuncVersionUrl (0.00s) --- PASS: TestRuncVersionUrl/1.1.0- (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm (0.00s) --- PASS: TestRuncVersionUrl/-amd64 (0.00s) --- PASS: TestRuncVersionUrl/-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionUrl/1.1.0-arm64 (0.00s) === RUN TestRuncVersionHash === RUN TestRuncVersionHash/1.1.0- === RUN TestRuncVersionHash/1.1.0-arm === RUN TestRuncVersionHash/-amd64 === RUN TestRuncVersionHash/-arm64 === RUN TestRuncVersionHash/1.0.0-amd64 === RUN TestRuncVersionHash/1.0.0-arm64 === RUN TestRuncVersionHash/1.100.0-amd64 === RUN TestRuncVersionHash/1.100.0-arm64 === RUN TestRuncVersionHash/1.1.0-amd64 === RUN TestRuncVersionHash/1.1.0-arm64 --- PASS: TestRuncVersionHash (0.00s) --- PASS: TestRuncVersionHash/1.1.0- (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm (0.00s) --- PASS: TestRuncVersionHash/-amd64 (0.00s) --- PASS: TestRuncVersionHash/-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.0.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.100.0-arm64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-amd64 (0.00s) --- PASS: TestRuncVersionHash/1.1.0-arm64 (0.00s) === RUN TestRuncVersionsHashesAmd64 runc_test.go:240: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesAmd64 (0.00s) === RUN TestRuncVersionsHashesArm64 runc_test.go:255: VERIFY_HASHES not set, won't download & verify runc hashes --- SKIP: TestRuncVersionsHashesArm64 (0.00s) === RUN Test_AssignSubnets I0216 02:06:30.457801 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.457828 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.457839 52318 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0216 02:06:30.457857 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0216 02:06:30.457868 52318 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet a I0216 02:06:30.457887 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet b I0216 02:06:30.457906 52318 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet b I0216 02:06:30.457929 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.457941 52318 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0216 02:06:30.457961 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.457973 52318 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0216 02:06:30.457991 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.458016 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.458036 52318 subnets.go:185] Assigned CIDR 10.32.0.0/11 to subnet a I0216 02:06:30.458048 52318 subnets.go:185] Assigned CIDR 10.64.0.0/11 to subnet b I0216 02:06:30.458058 52318 subnets.go:185] Assigned CIDR 10.96.0.0/11 to subnet b I0216 02:06:30.458068 52318 subnets.go:185] Assigned CIDR 10.128.0.0/11 to subnet c I0216 02:06:30.458077 52318 subnets.go:185] Assigned CIDR 10.160.0.0/11 to subnet c I0216 02:06:30.458087 52318 subnets.go:199] Assigned CIDR 10.0.0.0/14 to subnet a I0216 02:06:30.458098 52318 subnets.go:199] Assigned CIDR 10.4.0.0/14 to subnet b I0216 02:06:30.458108 52318 subnets.go:199] Assigned CIDR 10.8.0.0/14 to subnet c --- PASS: Test_AssignSubnets (0.00s) === RUN Test_TemplateFunctions_CloudControllerConfigArgv === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration === RUN Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Log_Level_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ExternalCloudControllerManager_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/No_CloudProvider_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/k8s_cluster_name (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Default_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Cluster-cidr_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/AllocateNodeCIDRs_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/ConfigureCloudRoutes_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/CIDRAllocatorType_Configuration#01 (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Election (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Leader_Migration (0.00s) --- PASS: Test_TemplateFunctions_CloudControllerConfigArgv/Disable_Controller (0.00s) === RUN Test_KarpenterInstanceTypes I0216 02:06:30.459271 52318 images.go:53] DescribeImages: { ImageIds: ["ami-073c8c0760395aab8"] } --- PASS: Test_KarpenterInstanceTypes (0.00s) === RUN Test_BuildMirroredAsset === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup === RUN Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup --- PASS: Test_BuildMirroredAsset (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/amd64/nodeup (0.00s) --- PASS: Test_BuildMirroredAsset/https://artifacts.k8s.io/binaries/kops/%s/linux/arm64/nodeup (0.00s) === RUN TestValidateFull_Default_Validates --- PASS: TestValidateFull_Default_Validates (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_NoDot --- PASS: TestValidateFull_ClusterName_InvalidDNS_NoDot (0.00s) === RUN TestValidateFull_ClusterName_InvalidDNS_Invalid --- PASS: TestValidateFull_ClusterName_InvalidDNS_Invalid (0.00s) === RUN TestValidateFull_ClusterName_Required --- PASS: TestValidateFull_ClusterName_Required (0.00s) === RUN TestValidateFull_UpdatePolicy_Valid === RUN TestValidateFull_UpdatePolicy_Valid/missing === RUN TestValidateFull_UpdatePolicy_Valid/automatic === RUN TestValidateFull_UpdatePolicy_Valid/external --- PASS: TestValidateFull_UpdatePolicy_Valid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/missing (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/automatic (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Valid/external (0.00s) === RUN TestValidateFull_UpdatePolicy_Invalid === RUN TestValidateFull_UpdatePolicy_Invalid/empty === RUN TestValidateFull_UpdatePolicy_Invalid/populated --- PASS: TestValidateFull_UpdatePolicy_Invalid (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/empty (0.00s) --- PASS: TestValidateFull_UpdatePolicy_Invalid/populated (0.00s) === RUN Test_Validate_Kubenet_With_14 --- PASS: Test_Validate_Kubenet_With_14 (0.00s) === RUN TestValidate_ClusterName_Import --- PASS: TestValidate_ClusterName_Import (0.00s) === RUN TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity --- PASS: TestValidate_ContainerRegistry_and_ContainerProxy_exclusivity (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup 1.698s === RUN TestGetASGTagsToDelete --- PASS: TestGetASGTagsToDelete (0.00s) === RUN TestProcessCompare --- PASS: TestProcessCompare (0.00s) === RUN TestAutoscalingGroupTerraformRender I0216 02:06:28.837392 52319 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2825637339/001 I0216 02:06:28.840348 52319 target.go:112] Terraform output is in /tmp/TestAutoscalingGroupTerraformRender2825637339/001 --- PASS: TestAutoscalingGroupTerraformRender (0.01s) === RUN TestTGsARNsChunks --- PASS: TestTGsARNsChunks (0.01s) === RUN TestGetEBSVolumeTagsToDelete --- PASS: TestGetEBSVolumeTagsToDelete (0.00s) === RUN TestSharedEgressOnlyInternetGatewayDoesNotRename I0216 02:06:28.848255 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0216 02:06:28.848420 52319 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0216 02:06:28.848508 52319 egressonlyinternetgateways.go:66] CreateEgressOnlyInternetGateway: { TagSpecifications: [{ ResourceType: "egress-only-internet-gateway", Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] }], VpcId: "vpc-1" } I0216 02:06:28.848814 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.849025 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.849134 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.849162 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.849406 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.849462 52319 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0216 02:06:28.849537 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.849640 52319 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0216 02:06:28.849759 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0216 02:06:28.849942 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.850001 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.850060 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.850084 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.850134 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.850188 52319 egressonlyinternetgateways.go:106] DescribeEgressOnlyInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0216 02:06:28.850271 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedEgressOnlyInternetGatewayDoesNotRename (0.00s) === RUN TestElasticIPCreate I0216 02:06:28.850710 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.850788 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.850971 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.851034 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.851070 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.851118 52319 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0216 02:06:28.851283 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.851531 52319 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0216 02:06:28.851604 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.851658 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.851699 52319 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0216 02:06:28.851772 52319 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.851979 52319 address.go:79] AllocateAddress: { Domain: "vpc", TagSpecifications: [{ ResourceType: "elastic-ip", Tags: [{ Key: "Name", Value: "eip1" }] }] } I0216 02:06:28.852061 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.852099 52319 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "AssociatedElasticIp", Value: "192.0.2.1" },{ Key: "AssociatedElasticIpAllocationId", Value: "eipalloc-1" }] } I0216 02:06:28.852157 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0216 02:06:28.852375 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.852438 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.852505 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.852530 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.852570 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.852607 52319 executor.go:111] Tasks: 1 done / 3 total; 1 can run I0216 02:06:28.852661 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.852740 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.852781 52319 executor.go:111] Tasks: 2 done / 3 total; 1 can run I0216 02:06:28.852835 52319 tags.go:165] DescribeTags { Filters: [{ Name: "key", Values: ["AssociatedElasticIp"] },{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.852907 52319 address.go:120] DescribeAddresses: { Filters: [{ Name: "public-ip", Values: ["192.0.2.1"] }] } I0216 02:06:28.852949 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["eipalloc-1"] }] } I0216 02:06:28.853017 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestElasticIPCreate (0.00s) === RUN TestSharedInternetGatewayDoesNotRename I0216 02:06:28.853250 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16" } I0216 02:06:28.853304 52319 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "Name", Value: "ExistingVPC" }] } I0216 02:06:28.853347 52319 internetgateways.go:66] CreateInternetGateway: { } I0216 02:06:28.853379 52319 tags.go:42] CreateTags { Resources: ["igw-1"], Tags: [{ Key: "Name", Value: "ExistingInternetGateway" }] } I0216 02:06:28.853554 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.853613 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.853667 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.853691 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.853803 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.853842 52319 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0216 02:06:28.853889 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.853987 52319 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0216 02:06:28.854094 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0216 02:06:28.854274 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.854333 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.854390 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.854416 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.854464 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.854517 52319 internetgateways.go:101] DescribeInternetGateways: { Filters: [{ Name: "attachment.vpc-id", Values: ["vpc-1"] }] } I0216 02:06:28.854599 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedInternetGatewayDoesNotRename (0.00s) === RUN TestLaunchTemplateTerraformRender I0216 02:06:28.857701 52319 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1690227723/001 I0216 02:06:28.859948 52319 target.go:112] Terraform output is in /tmp/TestLaunchTemplateTerraformRender1690227723/001 --- PASS: TestLaunchTemplateTerraformRender (0.01s) === RUN TestParseRemovalRule --- PASS: TestParseRemovalRule (0.00s) === RUN TestPortRemovalRule --- PASS: TestPortRemovalRule (0.00s) === RUN TestPortRemovalRule_Zero --- PASS: TestPortRemovalRule_Zero (0.00s) === RUN TestSecurityGroupCreate I0216 02:06:28.860844 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.860928 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.861096 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.861167 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.861205 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.861237 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.861326 52319 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0216 02:06:28.861608 52319 securitygroups.go:40] CreateSecurityGroup: { Description: "Description", GroupName: "sg1", TagSpecifications: [{ ResourceType: "security-group", Tags: [{ Key: "Name", Value: "sg1" }] }], VpcId: "vpc-1" } I0216 02:06:28.861669 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["sg-1"] }] } I0216 02:06:28.861714 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0216 02:06:28.861888 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.861954 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.862021 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.862048 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.862089 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.862124 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.862182 52319 securitygroups.go:115] DescribeSecurityGroups: { Filters: [{ Name: "tag:Name", Values: ["sg1"] },{ Name: "vpc-id", Values: ["vpc-1"] },{ Name: "group-name", Values: ["sg1"] }] } I0216 02:06:28.862288 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSecurityGroupCreate (0.00s) === RUN Test_Subnet_ValidateRequired --- PASS: Test_Subnet_ValidateRequired (0.00s) === RUN Test_Subnet_CannotChangeSubnet --- PASS: Test_Subnet_CannotChangeSubnet (0.00s) === RUN TestSubnetCreate I0216 02:06:28.862801 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.862861 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.862988 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.863045 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.863078 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.863112 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.863171 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.863326 52319 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0216 02:06:28.863391 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.863435 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.863472 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0216 02:06:28.863645 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.863710 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.863776 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.863801 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.863851 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.863888 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.863945 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.864036 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.864073 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSubnetCreate (0.00s) === RUN TestSubnetCreateIPv6 I0216 02:06:28.864457 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.864520 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.864649 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.864707 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.864742 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.864776 52319 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0216 02:06:28.864903 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.864994 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.865252 52319 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:28.865295 52319 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0216 02:06:28.865424 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.865483 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.865524 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0216 02:06:28.865830 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.865902 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.865974 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.865998 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.866036 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.866073 52319 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0216 02:06:28.866139 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.866219 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.866302 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.866337 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6 (0.00s) === RUN TestSubnetCreateIPv6NetNum I0216 02:06:28.866756 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.866825 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.866969 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.867036 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.867076 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.867112 52319 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0216 02:06:28.867169 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.867308 52319 vpcs.go:246] AssociateVpcCidrBlock: { AmazonProvidedIpv6CidrBlock: true, VpcId: "vpc-1" } I0216 02:06:28.867374 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.867663 52319 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", Ipv6CidrBlock: "2001:db8:0:1::/64", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "subnet1" }] }], VpcId: "vpc-1" } I0216 02:06:28.867737 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.867775 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.867810 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run I0216 02:06:28.867993 52319 executor.go:111] Tasks: 0 done / 3 total; 1 can run I0216 02:06:28.868056 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.868128 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.868155 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.868192 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.868248 52319 executor.go:111] Tasks: 1 done / 3 total; 2 can run I0216 02:06:28.868322 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.868395 52319 subnets.go:128] DescribeSubnets: { Filters: [{ Name: "tag:Name", Values: ["subnet1"] }] } I0216 02:06:28.868522 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.868569 52319 executor.go:111] Tasks: 3 done / 3 total; 0 can run --- PASS: TestSubnetCreateIPv6NetNum (0.00s) === RUN TestSharedSubnetCreateDoesNotCreateNew I0216 02:06:28.868779 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.20.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "ExistingVPC" }] }] } I0216 02:06:28.868882 52319 subnets.go:110] CreateSubnet: { CidrBlock: "172.20.1.0/24", TagSpecifications: [{ ResourceType: "subnet", Tags: [{ Key: "Name", Value: "ExistingSubnet" }] }], VpcId: "vpc-1" } I0216 02:06:28.869074 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.869134 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.869211 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.869236 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.869343 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.869381 52319 tags.go:42] CreateTags { Resources: ["vpc-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0216 02:06:28.869428 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.869482 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.869673 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["subnet-1"] }] } I0216 02:06:28.869711 52319 tags.go:42] CreateTags { Resources: ["subnet-1"], Tags: [{ Key: "kubernetes.io/cluster/cluster.example.com", Value: "shared" }] } I0216 02:06:28.869754 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run I0216 02:06:28.869953 52319 executor.go:111] Tasks: 0 done / 2 total; 1 can run I0216 02:06:28.870014 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.870076 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.870140 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.870189 52319 executor.go:111] Tasks: 1 done / 2 total; 1 can run I0216 02:06:28.870244 52319 subnets.go:128] DescribeSubnets: { SubnetIds: ["subnet-1"] } I0216 02:06:28.870337 52319 executor.go:111] Tasks: 2 done / 2 total; 0 can run --- PASS: TestSharedSubnetCreateDoesNotCreateNew (0.00s) === RUN TestVPCCreate I0216 02:06:28.870671 52319 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:28.870732 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.870879 52319 vpcs.go:89] CreateVpc: { CidrBlock: "172.21.0.0/16", TagSpecifications: [{ ResourceType: "vpc", Tags: [{ Key: "Name", Value: "vpc1" }] }] } I0216 02:06:28.870934 52319 tags.go:165] DescribeTags { Filters: [{ Name: "resource-id", Values: ["vpc-1"] }] } I0216 02:06:28.870967 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.871016 52319 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:28.871165 52319 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:28.871237 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc1"] }] } I0216 02:06:28.871313 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.871338 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.871388 52319 vpcs.go:112] DescribeVpcs: { VpcIds: ["vpc-1"] } I0216 02:06:28.871424 52319 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestVPCCreate (0.00s) === RUN Test4758 --- PASS: Test4758 (0.00s) === RUN TestSharedVPCAdditionalCIDR I0216 02:06:28.871713 52319 vpcs.go:246] AssociateVpcCidrBlock: { CidrBlock: "172.22.0.0/16", VpcId: "vpc-1" } I0216 02:06:28.871891 52319 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:28.871953 52319 vpcs.go:112] DescribeVpcs: { Filters: [{ Name: "tag:Name", Values: ["vpc-1"] }] } I0216 02:06:28.872030 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsSupport", VpcId: "vpc-1" } I0216 02:06:28.872058 52319 vpcs.go:169] DescribeVpcAttribute: { Attribute: "enableDnsHostnames", VpcId: "vpc-1" } I0216 02:06:28.872117 52319 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestSharedVPCAdditionalCIDR (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awstasks 0.093s === RUN TestValidateRegion --- PASS: TestValidateRegion (0.00s) === RUN TestFindRegion --- PASS: TestFindRegion (0.00s) === RUN TestEC2TagSpecification === RUN TestEC2TagSpecification/No_tags === RUN TestEC2TagSpecification/simple_tag --- PASS: TestEC2TagSpecification (0.00s) --- PASS: TestEC2TagSpecification/No_tags (0.00s) --- PASS: TestEC2TagSpecification/simple_tag (0.00s) === RUN Test_GetResourceName32 --- PASS: Test_GetResourceName32 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/awsup 0.051s === RUN TestZoneToLocation === RUN TestZoneToLocation/test_case_0 === RUN TestZoneToLocation/test_case_1 === RUN TestZoneToLocation/test_case_2 --- PASS: TestZoneToLocation (0.00s) --- PASS: TestZoneToLocation/test_case_0 (0.00s) --- PASS: TestZoneToLocation/test_case_1 (0.00s) --- PASS: TestZoneToLocation/test_case_2 (0.00s) === RUN TestZoneToAvailabilityZoneNumber === RUN TestZoneToAvailabilityZoneNumber/test_case_0 === RUN TestZoneToAvailabilityZoneNumber/test_case_1 === RUN TestZoneToAvailabilityZoneNumber/test_case_2 --- PASS: TestZoneToAvailabilityZoneNumber (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_0 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_1 (0.00s) --- PASS: TestZoneToAvailabilityZoneNumber/test_case_2 (0.00s) === RUN TestFindEtcdStatus --- PASS: TestFindEtcdStatus (0.00s) === RUN TestGetCloudGroups --- PASS: TestGetCloudGroups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azure 0.024s === RUN TestDiskRenderAzure I0216 02:06:29.065526 52488 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRenderAzure (0.00s) === RUN TestDiskFind --- PASS: TestDiskFind (0.00s) === RUN TestDiskRun I0216 02:06:29.066263 52488 disk.go:105] Creating a new Disk with name: disk --- PASS: TestDiskRun (0.00s) === RUN TestDiskCheckChanges === RUN TestDiskCheckChanges/test_case_0 === RUN TestDiskCheckChanges/test_case_1 === RUN TestDiskCheckChanges/test_case_2 === RUN TestDiskCheckChanges/test_case_3 --- PASS: TestDiskCheckChanges (0.00s) --- PASS: TestDiskCheckChanges/test_case_0 (0.00s) --- PASS: TestDiskCheckChanges/test_case_1 (0.00s) --- PASS: TestDiskCheckChanges/test_case_2 (0.00s) --- PASS: TestDiskCheckChanges/test_case_3 (0.00s) === RUN TestLoadBalancerRenderAzure I0216 02:06:29.066662 52488 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRenderAzure (0.00s) === RUN TestLoadBalancerFind loadbalancer_test.go:122: *azuretasks.LoadBalancer {"Name":"loadbalancer","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Subnet":{"Name":"subnet","Lifecycle":"","ResourceGroup":null,"VirtualNetwork":null,"CIDR":null,"Shared":null},"External":true,"Tags":null,"ForAPIServer":false} --- PASS: TestLoadBalancerFind (0.00s) === RUN TestLoadBalancerRun I0216 02:06:29.067163 52488 loadbalancer.go:127] Creating a new Loadbalancer with name: loadbalancer --- PASS: TestLoadBalancerRun (0.00s) === RUN TestLoadBalancerCheckChanges === RUN TestLoadBalancerCheckChanges/test_case_0 === RUN TestLoadBalancerCheckChanges/test_case_1 === RUN TestLoadBalancerCheckChanges/test_case_2 === RUN TestLoadBalancerCheckChanges/test_case_3 --- PASS: TestLoadBalancerCheckChanges (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_0 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_1 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_2 (0.00s) --- PASS: TestLoadBalancerCheckChanges/test_case_3 (0.00s) === RUN TestPublicIPAddressRenderAzure I0216 02:06:29.067460 52488 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRenderAzure (0.00s) === RUN TestPublicIPAddressFind publicipaddress_test.go:100: *azuretasks.PublicIPAddress {"Name":"publicIPAddress","Lifecycle":"","ResourceGroup":{"Name":"rg","Lifecycle":"","Tags":null,"Shared":null},"Tags":null} --- PASS: TestPublicIPAddressFind (0.00s) === RUN TestPublicIPAddressRun I0216 02:06:29.067743 52488 publicipaddress.go:104] Creating a new Public IP Address with name: publicIPAddress --- PASS: TestPublicIPAddressRun (0.00s) === RUN TestPublicIPAddressCheckChanges === RUN TestPublicIPAddressCheckChanges/test_case_0 === RUN TestPublicIPAddressCheckChanges/test_case_1 === RUN TestPublicIPAddressCheckChanges/test_case_2 === RUN TestPublicIPAddressCheckChanges/test_case_3 --- PASS: TestPublicIPAddressCheckChanges (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_0 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_1 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_2 (0.00s) --- PASS: TestPublicIPAddressCheckChanges/test_case_3 (0.00s) === RUN TestResourceGroupRenderAzure I0216 02:06:29.068003 52488 resourcegroup.go:103] Creating a new Resource Group with name: rg I0216 02:06:29.068018 52488 resourcegroup.go:105] Updating a Resource Group with name: rg --- PASS: TestResourceGroupRenderAzure (0.00s) === RUN TestResourceGroupFind --- PASS: TestResourceGroupFind (0.00s) === RUN TestResourceGroupRun I0216 02:06:29.068204 52488 resourcegroup.go:103] Creating a new Resource Group with name: rg --- PASS: TestResourceGroupRun (0.00s) === RUN TestResourceGroupCheckChanges === RUN TestResourceGroupCheckChanges/test_case_0 === RUN TestResourceGroupCheckChanges/test_case_1 === RUN TestResourceGroupCheckChanges/test_case_2 === RUN TestResourceGroupCheckChanges/test_case_3 --- PASS: TestResourceGroupCheckChanges (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_0 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_1 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_2 (0.00s) --- PASS: TestResourceGroupCheckChanges/test_case_3 (0.00s) === RUN TestRoleAssignmentRenderAzure --- PASS: TestRoleAssignmentRenderAzure (0.00s) === RUN TestRoleAssignmentFind --- PASS: TestRoleAssignmentFind (0.00s) === RUN TestRoleAssignmentFind_NoPrincipalID --- PASS: TestRoleAssignmentFind_NoPrincipalID (0.00s) === RUN TestRoleAssignmentCheckChanges === RUN TestRoleAssignmentCheckChanges/test_case_0 === RUN TestRoleAssignmentCheckChanges/test_case_1 === RUN TestRoleAssignmentCheckChanges/test_case_2 === RUN TestRoleAssignmentCheckChanges/test_case_3 --- PASS: TestRoleAssignmentCheckChanges (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_0 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_1 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_2 (0.00s) --- PASS: TestRoleAssignmentCheckChanges/test_case_3 (0.00s) === RUN TestSubnetRenderAzure I0216 02:06:29.068839 52488 subnet.go:106] Creating a new Subnet with name: vnet --- PASS: TestSubnetRenderAzure (0.00s) === RUN TestSubnetFind --- PASS: TestSubnetFind (0.00s) === RUN TestSubnetCheckChanges === RUN TestSubnetCheckChanges/test_case_0 === RUN TestSubnetCheckChanges/test_case_1 === RUN TestSubnetCheckChanges/test_case_2 === RUN TestSubnetCheckChanges/test_case_3 --- PASS: TestSubnetCheckChanges (0.00s) --- PASS: TestSubnetCheckChanges/test_case_0 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_1 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_2 (0.00s) --- PASS: TestSubnetCheckChanges/test_case_3 (0.00s) === RUN TestVirtualNetworkRenderAzure I0216 02:06:29.069181 52488 virtualnetwork.go:116] Creating a new Virtual Network with name: vnet --- PASS: TestVirtualNetworkRenderAzure (0.00s) === RUN TestVirtualNetworkFind --- PASS: TestVirtualNetworkFind (0.00s) === RUN TestVirtualNetworkRun I0216 02:06:29.069486 52488 virtualnetwork.go:116] Creating a new Virtual Network with name: rg --- PASS: TestVirtualNetworkRun (0.00s) === RUN TestVirtualNetworkCheckChanges === RUN TestVirtualNetworkCheckChanges/test_case_0 === RUN TestVirtualNetworkCheckChanges/test_case_1 === RUN TestVirtualNetworkCheckChanges/test_case_2 === RUN TestVirtualNetworkCheckChanges/test_case_3 --- PASS: TestVirtualNetworkCheckChanges (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_0 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_1 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_2 (0.00s) --- PASS: TestVirtualNetworkCheckChanges/test_case_3 (0.00s) === RUN TestSubnetIDParse --- PASS: TestSubnetIDParse (0.00s) === RUN TestLoadBalancerIDParse --- PASS: TestLoadBalancerIDParse (0.00s) === RUN TestVMScaleSetRenderAzure I0216 02:06:29.070019 52488 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRenderAzure (0.00s) === RUN TestVMScaleSetFind --- PASS: TestVMScaleSetFind (0.00s) === RUN TestVMScaleSetRun I0216 02:06:29.070334 52488 vmscaleset.go:269] Creating a new VM Scale Set with name: vmss --- PASS: TestVMScaleSetRun (0.00s) === RUN TestVMScaleSetCheckChanges === RUN TestVMScaleSetCheckChanges/test_case_0 === RUN TestVMScaleSetCheckChanges/test_case_1 === RUN TestVMScaleSetCheckChanges/test_case_2 === RUN TestVMScaleSetCheckChanges/test_case_3 --- PASS: TestVMScaleSetCheckChanges (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_0 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_1 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_2 (0.00s) --- PASS: TestVMScaleSetCheckChanges/test_case_3 (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/azuretasks 0.037s === RUN Test_Find === RUN Test_Find/successfully_found_volume === RUN Test_Find/no_volume_found === RUN Test_Find/error_from_server --- PASS: Test_Find (0.00s) --- PASS: Test_Find/successfully_found_volume (0.00s) --- PASS: Test_Find/no_volume_found (0.00s) --- PASS: Test_Find/error_from_server (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/dotasks 0.032s ? k8s.io/kops/upup/pkg/fi/cloudup/hetzner [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/hetznertasks [no test files] ? k8s.io/kops/upup/pkg/fi/cloudup/spotinsttasks [no test files] ? k8s.io/kops/upup/pkg/fi/loader [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/cloudinit [no test files] ? k8s.io/kops/upup/pkg/fi/nodeup/local [no test files] === RUN TestProjectIAMBinding I0216 02:06:32.109480 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.110131 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.110283 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.110797 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.110942 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.111103 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestProjectIAMBinding (0.00s) === RUN TestServiceAccount I0216 02:06:32.111347 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.111440 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.111574 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.111742 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.111874 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.111930 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestServiceAccount (0.00s) === RUN TestStorageBucketIAM I0216 02:06:32.112174 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.112421 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.112551 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.112885 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run I0216 02:06:32.113024 53441 executor.go:111] Tasks: 0 done / 1 total; 1 can run I0216 02:06:32.113158 53441 executor.go:111] Tasks: 1 done / 1 total; 0 can run --- PASS: TestStorageBucketIAM (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/gcetasks 0.060s === RUN Test_OpenstackCloud_GetApiIngressStatus === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match === RUN Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled === RUN Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled --- PASS: Test_OpenstackCloud_GetApiIngressStatus (0.01s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_set_multiple_IPs_match (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/Loadbalancer_configured_master_public_name_not_set (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_no_floating_enabled (0.00s) --- PASS: Test_OpenstackCloud_GetApiIngressStatus/No_Loadbalancer_configured_with_floating_enabled (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstack 0.048s === RUN TestFilterPortsReturnsAllPorts --- PASS: TestFilterPortsReturnsAllPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPort --- PASS: TestFilterPortsReturnsOnlyTaggedPort (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPorts --- PASS: TestFilterPortsReturnsOnlyTaggedPorts (0.00s) === RUN TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername --- PASS: TestFilterPortsReturnsOnlyTaggedPortsWithLongClustername (0.00s) === RUN Test_Port_GetDependencies --- PASS: Test_Port_GetDependencies (0.00s) === RUN Test_NewPortTaskFromCloud === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName === RUN Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found === RUN Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups --- PASS: Test_NewPortTaskFromCloud (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/empty_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_not_nil_populates_the_InstanceGroupName (0.00s) --- PASS: Test_NewPortTaskFromCloud/fully_populated_cloud_port_found_port_nil_populates_the_InstanceGroupName_if_found (0.00s) --- PASS: Test_NewPortTaskFromCloud/cloud_port_found_port_not_nil_honors_additional_security_groups (0.00s) === RUN Test_Port_Find === RUN Test_Port_Find/nothing_found === RUN Test_Port_Find/port_found_no_tags === RUN Test_Port_Find/port_found_with_tags === RUN Test_Port_Find/multiple_ports_found === RUN Test_Port_Find/error_listing_ports --- PASS: Test_Port_Find (0.00s) --- PASS: Test_Port_Find/nothing_found (0.00s) --- PASS: Test_Port_Find/port_found_no_tags (0.00s) --- PASS: Test_Port_Find/port_found_with_tags (0.00s) --- PASS: Test_Port_Find/multiple_ports_found (0.00s) --- PASS: Test_Port_Find/error_listing_ports (0.00s) === RUN Test_Port_CheckChanges === RUN Test_Port_CheckChanges/actual_nil_all_required_fields_set === RUN Test_Port_CheckChanges/actual_nil_required_field_Name_nil === RUN Test_Port_CheckChanges/actual_nil_required_field_Network_nil === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set === RUN Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set === RUN Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set --- PASS: Test_Port_CheckChanges (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_all_required_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Name_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_nil_required_field_Network_nil (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_all_changeable_fields_set#01 (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Name_set (0.00s) --- PASS: Test_Port_CheckChanges/actual_not_nil_unchangeable_field_Network_set (0.00s) === RUN Test_Port_RenderOpenstack === RUN Test_Port_RenderOpenstack/actual_not_nil === RUN Test_Port_RenderOpenstack/actual_nil_success === RUN Test_Port_RenderOpenstack/actual_nil_cloud_error --- PASS: Test_Port_RenderOpenstack (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_not_nil (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_success (0.00s) --- PASS: Test_Port_RenderOpenstack/actual_nil_cloud_error (0.00s) === RUN Test_Port_createOptsFromPortTask === RUN Test_Port_createOptsFromPortTask/all_fields_set === RUN Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups --- PASS: Test_Port_createOptsFromPortTask (0.00s) --- PASS: Test_Port_createOptsFromPortTask/all_fields_set (0.00s) --- PASS: Test_Port_createOptsFromPortTask/nonexisting_additional_security_groups (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/openstacktasks 0.030s === RUN TestWriteValue === RUN TestWriteValue/null === RUN TestWriteValue/empty_list === RUN TestWriteValue/string === RUN TestWriteValue/list === RUN TestWriteValue/list_of_objects === RUN TestWriteValue/object_block --- PASS: TestWriteValue (0.00s) --- PASS: TestWriteValue/null (0.00s) --- PASS: TestWriteValue/empty_list (0.00s) --- PASS: TestWriteValue/string (0.00s) --- PASS: TestWriteValue/list (0.00s) --- PASS: TestWriteValue/list_of_objects (0.00s) --- PASS: TestWriteValue/object_block (0.00s) === RUN TestWriteLiteral === RUN TestWriteLiteral/string === RUN TestWriteLiteral/traversal === RUN TestWriteLiteral/provider_alias === RUN TestWriteLiteral/file --- PASS: TestWriteLiteral (0.00s) --- PASS: TestWriteLiteral/string (0.00s) --- PASS: TestWriteLiteral/traversal (0.00s) --- PASS: TestWriteLiteral/provider_alias (0.00s) --- PASS: TestWriteLiteral/file (0.00s) === RUN TestWriteLiteralList === RUN TestWriteLiteralList/empty_list === RUN TestWriteLiteralList/one_literal === RUN TestWriteLiteralList/two_literals === RUN TestWriteLiteralList/one_traversal_literal,_one_string_literal --- PASS: TestWriteLiteralList (0.00s) --- PASS: TestWriteLiteralList/empty_list (0.00s) --- PASS: TestWriteLiteralList/one_literal (0.00s) --- PASS: TestWriteLiteralList/two_literals (0.00s) --- PASS: TestWriteLiteralList/one_traversal_literal,_one_string_literal (0.00s) === RUN TestWriteMap === RUN TestWriteMap/empty_map === RUN TestWriteMap/simple_map === RUN TestWriteMap/complex_keys === RUN TestWriteMap/literal_list --- PASS: TestWriteMap (0.00s) --- PASS: TestWriteMap/empty_map (0.00s) --- PASS: TestWriteMap/simple_map (0.00s) --- PASS: TestWriteMap/complex_keys (0.00s) --- PASS: TestWriteMap/literal_list (0.00s) === RUN TestWriteMapLiterals === RUN TestWriteMapLiterals/literal_values --- PASS: TestWriteMapLiterals (0.00s) --- PASS: TestWriteMapLiterals/literal_values (0.00s) === RUN TestWriteLocalsOutputs === RUN TestWriteLocalsOutputs/empty_map === RUN TestWriteLocalsOutputs/single_output === RUN TestWriteLocalsOutputs/list_output --- PASS: TestWriteLocalsOutputs (0.00s) --- PASS: TestWriteLocalsOutputs/empty_map (0.00s) --- PASS: TestWriteLocalsOutputs/single_output (0.00s) --- PASS: TestWriteLocalsOutputs/list_output (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraform 0.032s === RUN TestGetOutputs === RUN TestGetOutputs/empty_map === RUN TestGetOutputs/single_output === RUN TestGetOutputs/list_output === RUN TestGetOutputs/duplicate_names === RUN TestGetOutputs/duplicate_values --- PASS: TestGetOutputs (0.00s) --- PASS: TestGetOutputs/empty_map (0.00s) --- PASS: TestGetOutputs/single_output (0.00s) --- PASS: TestGetOutputs/list_output (0.00s) --- PASS: TestGetOutputs/duplicate_names (0.00s) --- PASS: TestGetOutputs/duplicate_values (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/cloudup/terraformWriter 0.006s === RUN TestKeypairDeps --- PASS: TestKeypairDeps (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/fitasks 0.027s ? k8s.io/kops/upup/pkg/fi/nodeup/nodetasks/dnstasks [no test files] ? k8s.io/kops/upup/pkg/fi/secrets [no test files] ? k8s.io/kops/upup/pkg/kutil [no test files] ? k8s.io/kops/upup/tools/generators/fitask [no test files] ? k8s.io/kops/util/pkg/architectures [no test files] ? k8s.io/kops/util/pkg/env [no test files] ? k8s.io/kops/util/pkg/mirrors [no test files] ? k8s.io/kops/util/pkg/tables [no test files] ? k8s.io/kops/util/pkg/ui [no test files] === RUN TestArchiveDependencies --- PASS: TestArchiveDependencies (0.00s) === RUN TestBindMountCommands I0216 02:06:35.670386 53709 bindmount.go:224] running mount command [mount --bind /containerized_mounter /containerized_mounter] I0216 02:06:35.670458 53709 bindmount.go:233] running mount command [mount -o remount,exec /containerized_mounter] I0216 02:06:35.670468 53709 bindmount.go:224] running mount command [mount --rbind /var/lib/kubelet/ /containerized_mounter/rootfs/var/lib/kubelet] I0216 02:06:35.670479 53709 bindmount.go:244] running mount command [mount --make-rshared /containerized_mounter/rootfs/var/lib/kubelet] I0216 02:06:35.670488 53709 bindmount.go:224] running mount command [mount --bind -o ro /proc /containerized_mounter/rootfs/proc] I0216 02:06:35.670496 53709 bindmount.go:224] running mount command [mount --bind -o ro /dev /containerized_mounter/rootfs/dev] I0216 02:06:35.670505 53709 bindmount.go:224] running mount command [mount --bind -o ro /etc/resolv.conf /containerized_mounter/rootfs/etc/resolv.conf] --- PASS: TestBindMountCommands (0.00s) === RUN TestBindMountDependencies --- PASS: TestBindMountDependencies (0.00s) === RUN TestFileDependencies === RUN TestFileDependencies/user === RUN TestFileDependencies/parentDir === RUN TestFileDependencies/afterFiles --- PASS: TestFileDependencies (0.00s) --- PASS: TestFileDependencies/user (0.00s) --- PASS: TestFileDependencies/parentDir (0.00s) --- PASS: TestFileDependencies/afterFiles (0.00s) === RUN TestIssueCertFileDependencies --- PASS: TestIssueCertFileDependencies (0.00s) === RUN TestLoadImageTask_Deps --- PASS: TestLoadImageTask_Deps (0.00s) === RUN TestServiceTask_Deps --- PASS: TestServiceTask_Deps (0.00s) === RUN TestServiceTask_UnknownTypes W0216 02:06:35.670908 53709 service.go:92] Unhandled type *nodetasks.FakeTask in Service::GetDependencies: &{} --- PASS: TestServiceTask_UnknownTypes (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/nodeup/nodetasks 0.051s === RUN TestStringSlicesEqual --- PASS: TestStringSlicesEqual (0.00s) === RUN TestStringSlicesEqualIgnoreOrder --- PASS: TestStringSlicesEqualIgnoreOrder (0.00s) === RUN TestHashString --- PASS: TestHashString (0.00s) === RUN TestSanitizeString --- PASS: TestSanitizeString (0.00s) PASS ok k8s.io/kops/upup/pkg/fi/utils 0.006s === RUN TestFindDistribution --- PASS: TestFindDistribution (0.00s) PASS ok k8s.io/kops/util/pkg/distributions 0.004s === RUN TestWithTee --- PASS: TestWithTee (0.00s) PASS ok k8s.io/kops/util/pkg/exec 0.005s === RUN TestNewHasher === RUN TestNewHasher/sha256 === RUN TestNewHasher/sha1 === RUN TestNewHasher/md5 --- PASS: TestNewHasher (0.00s) --- PASS: TestNewHasher/sha256 (0.00s) --- PASS: TestNewHasher/sha1 (0.00s) --- PASS: TestNewHasher/md5 (0.00s) === RUN TestMemberFromString === RUN TestMemberFromString/sha256_1 === RUN TestMemberFromString/sha256_2 === RUN TestMemberFromString/sha256_3 === RUN TestMemberFromString/sha1_1 === RUN TestMemberFromString/sha1_2 === RUN TestMemberFromString/sha1_3 === RUN TestMemberFromString/md5_1 === RUN TestMemberFromString/md5_2 === RUN TestMemberFromString/md5_3 --- PASS: TestMemberFromString (0.00s) --- PASS: TestMemberFromString/sha256_1 (0.00s) --- PASS: TestMemberFromString/sha256_2 (0.00s) --- PASS: TestMemberFromString/sha256_3 (0.00s) --- PASS: TestMemberFromString/sha1_1 (0.00s) --- PASS: TestMemberFromString/sha1_2 (0.00s) --- PASS: TestMemberFromString/sha1_3 (0.00s) --- PASS: TestMemberFromString/md5_1 (0.00s) --- PASS: TestMemberFromString/md5_2 (0.00s) --- PASS: TestMemberFromString/md5_3 (0.00s) === RUN TestFromString === RUN TestFromString/sha256 === RUN TestFromString/sha1 === RUN TestFromString/md5 === RUN TestFromString/unknown --- PASS: TestFromString (0.00s) --- PASS: TestFromString/sha256 (0.00s) --- PASS: TestFromString/sha1 (0.00s) --- PASS: TestFromString/md5 (0.00s) --- PASS: TestFromString/unknown (0.00s) PASS ok k8s.io/kops/util/pkg/hashing 0.004s === RUN TestKeys --- PASS: TestKeys (0.00s) === RUN TestSortedKeys --- PASS: TestSortedKeys (0.00s) PASS ok k8s.io/kops/util/pkg/maps 0.006s === RUN TestGetProxyEnvVars W0216 02:06:35.646694 53714 proxy.go:34] EgressProxy set but no proxy host provided --- PASS: TestGetProxyEnvVars (0.00s) PASS ok k8s.io/kops/util/pkg/proxy 0.025s === RUN TestSet === RUN TestSet/simple_setting === RUN TestSet/setting_with_wildcard === RUN TestSet/creating_missing_objects === RUN TestSet/set_int === RUN TestSet/set_int32 === RUN TestSet/set_int64 === RUN TestSet/set_int_pointer === RUN TestSet/set_int32_pointer === RUN TestSet/set_int64_pointer === RUN TestSet/set_enum === RUN TestSet/set_enum_slice === RUN TestSet/append_enum_slice --- PASS: TestSet (0.00s) --- PASS: TestSet/simple_setting (0.00s) --- PASS: TestSet/setting_with_wildcard (0.00s) --- PASS: TestSet/creating_missing_objects (0.00s) --- PASS: TestSet/set_int (0.00s) --- PASS: TestSet/set_int32 (0.00s) --- PASS: TestSet/set_int64 (0.00s) --- PASS: TestSet/set_int_pointer (0.00s) --- PASS: TestSet/set_int32_pointer (0.00s) --- PASS: TestSet/set_int64_pointer (0.00s) --- PASS: TestSet/set_enum (0.00s) --- PASS: TestSet/set_enum_slice (0.00s) --- PASS: TestSet/append_enum_slice (0.00s) === RUN TestSetInvalidPath === RUN TestSetInvalidPath/setting_with_wildcard === RUN TestSetInvalidPath/creating_missing_objects === RUN TestSetInvalidPath/set_int === RUN TestSetInvalidPath/set_int32 === RUN TestSetInvalidPath/set_int64 --- PASS: TestSetInvalidPath (0.00s) --- PASS: TestSetInvalidPath/setting_with_wildcard (0.00s) --- PASS: TestSetInvalidPath/creating_missing_objects (0.00s) --- PASS: TestSetInvalidPath/set_int (0.00s) --- PASS: TestSetInvalidPath/set_int32 (0.00s) --- PASS: TestSetInvalidPath/set_int64 (0.00s) === RUN TestUnset === RUN TestUnset/simple_unsetting === RUN TestUnset/unsetting_with_wildcard === RUN TestUnset/uset_int === RUN TestUnset/unset_int32 === RUN TestUnset/unset_int64 === RUN TestUnset/unset_int_pointer === RUN TestUnset/unset_int32_pointer === RUN TestUnset/unset_int64_pointer === RUN TestUnset/unset_enum === RUN TestUnset/unset_enum_slice --- PASS: TestUnset (0.00s) --- PASS: TestUnset/simple_unsetting (0.00s) --- PASS: TestUnset/unsetting_with_wildcard (0.00s) --- PASS: TestUnset/uset_int (0.00s) --- PASS: TestUnset/unset_int32 (0.00s) --- PASS: TestUnset/unset_int64 (0.00s) --- PASS: TestUnset/unset_int_pointer (0.00s) --- PASS: TestUnset/unset_int32_pointer (0.00s) --- PASS: TestUnset/unset_int64_pointer (0.00s) --- PASS: TestUnset/unset_enum (0.00s) --- PASS: TestUnset/unset_enum_slice (0.00s) === RUN TestUnsetInvalidPath === RUN TestUnsetInvalidPath/usetting_with_wildcard === RUN TestUnsetInvalidPath/missing_objects --- PASS: TestUnsetInvalidPath (0.00s) --- PASS: TestUnsetInvalidPath/usetting_with_wildcard (0.00s) --- PASS: TestUnsetInvalidPath/missing_objects (0.00s) === RUN TestParseFieldPath === RUN TestParseFieldPath/test_Spec.Containers.Image === RUN TestParseFieldPath/test_Spec.Containers[0].Image === RUN TestParseFieldPath/test_Spec.Containers[*].Image --- PASS: TestParseFieldPath (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers.Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[0].Image (0.00s) --- PASS: TestParseFieldPath/test_Spec.Containers[*].Image (0.00s) === RUN Test_Merge --- PASS: Test_Merge (0.00s) PASS ok k8s.io/kops/util/pkg/reflectutils 0.007s === RUN TestGetUniqueStrings --- PASS: TestGetUniqueStrings (0.00s) === RUN TestContains --- PASS: TestContains (0.00s) PASS ok k8s.io/kops/util/pkg/slice 0.006s === RUN TestSplitToSections --- PASS: TestSplitToSections (0.00s) PASS ok k8s.io/kops/util/pkg/text 0.006s === RUN TestAzureBlobPathBase === RUN TestAzureBlobPathBase/Test_case_0 === RUN TestAzureBlobPathBase/Test_case_1 === RUN TestAzureBlobPathBase/Test_case_2 --- PASS: TestAzureBlobPathBase (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathBase/Test_case_2 (0.00s) === RUN TestAzureBlobPathPath === RUN TestAzureBlobPathPath/Test_case_0 === RUN TestAzureBlobPathPath/Test_case_1 === RUN TestAzureBlobPathPath/Test_case_2 --- PASS: TestAzureBlobPathPath (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_0 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_1 (0.00s) --- PASS: TestAzureBlobPathPath/Test_case_2 (0.00s) === RUN TestAzureBlobPathJoin --- PASS: TestAzureBlobPathJoin (0.00s) === RUN TestBuildVaultPath context_test.go:27: No vault dev token set --- SKIP: TestBuildVaultPath (0.00s) === RUN TestCreateFile --- PASS: TestCreateFile (0.00s) === RUN TestWriteTo --- PASS: TestWriteTo (0.00s) === RUN TestMemFsCreateFile --- PASS: TestMemFsCreateFile (0.00s) === RUN TestMemFsReadDir --- PASS: TestMemFsReadDir (0.00s) === RUN TestMemFsReadTree --- PASS: TestMemFsReadTree (0.00s) === RUN Test_VFSPath --- PASS: Test_VFSPath (0.00s) === RUN Test_S3Path_Parse --- PASS: Test_S3Path_Parse (0.00s) === RUN Test_newVaultPath vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_newVaultPath (0.00s) === RUN Test_newVaultPathHostOnly --- PASS: Test_newVaultPathHostOnly (0.00s) === RUN Test_WriteFileReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileReadFile (0.00s) === RUN Test_WriteFileDeleteFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_WriteFileDeleteFile (0.00s) === RUN Test_CreateFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_CreateFile (0.00s) === RUN Test_ReadFile vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadFile (0.00s) === RUN Test_Join vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_Join (0.00s) === RUN Test_VaultReadDirList vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadDirList (0.00s) === RUN Test_ReadDir vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_ReadDir (0.00s) === RUN Test_IsDirectory vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_IsDirectory (0.00s) === RUN Test_VaultReadTree vaultfs_test.go:36: No vault dev token set. Skipping --- SKIP: Test_VaultReadTree (0.00s) PASS ok k8s.io/kops/util/pkg/vfs 0.027s === RUN TestGSRenderTerraform === RUN TestGSRenderTerraform/gs://foo/bar --- PASS: TestGSRenderTerraform (0.00s) --- PASS: TestGSRenderTerraform/gs://foo/bar (0.00s) === RUN TestS3RenderTerraform === RUN TestS3RenderTerraform/s3://foo/bar W0216 02:06:38.405759 53801 s3context.go:239] Unable to read bucket encryption policy for "foo" in region "us-east-1": will encrypt using AES256 --- PASS: TestS3RenderTerraform (0.00s) --- PASS: TestS3RenderTerraform/s3://foo/bar (0.00s) PASS ok k8s.io/kops/util/pkg/vfs/tests 0.056s >>> kops: Entering fakeroot... >>> kops*: Running postcheck for kops >>> kops*: Preparing package kops... >>> kops*: Stripping binaries >>> kops*: Scanning shared objects >>> kops*: Tracing dependencies... >>> kops*: Package size: 134.3 MB >>> kops*: Compressing data... >>> kops*: Create checksum... >>> kops*: Create kops-1.25.3-r3.apk >>> kops: Build complete at Thu, 16 Feb 2023 02:06:42 +0000 elapsed time 0h 3m 27s >>> kops: Cleaning up srcdir >>> kops: Cleaning up pkgdir >>> kops: Uninstalling dependencies... (1/3) Purging .makedepends-kops (20230216.020317) (2/3) Purging kubernetes (1.26.1-r2) (3/3) Purging go (1.20.1-r0) Executing busybox-1.36.0-r4.trigger OK: 392 MiB in 98 packages >>> kops: Updating the testing/x86_64 repository index... >>> kops: Signing the index...